11 Feb 2021

feedPlanet Maemo

reBounce - softfp-to-hardfp LD_PRELOAD hack for Bounce on N9

This depends on Bounce (the N900 .deb) and SDL 1.2 being installed. Google "bounce_1.0.0_armel.deb" for the former, and use n9repomirror for the latter.

This is available on OpenRepos: https://openrepos.net/content/thp/rebounce
Also on TMO: https://talk.maemo.org/showthread.php?t=101160
Also on Twitter: https://twitter.com/thp4/status/1359758278620758019

Source code (copy'n'paste this into a file named "rebounce.c", then run it using your shell):

#if 0

gcc -Os -shared -fPIC -lSDL -o librebounce.so rebounce.c

LD_PRELOAD=$(pwd)/librebounce.so /opt/bounce/bin/bounce

exit 0

#endif


/**

* reBounce -- softfp-to-hardfp LD_PRELOAD hack for Bounce on N9

*

* Bounce was a really nice 2009 tech demo on the N900. This

* makes this tech demo work on an N9 by translating the calls

* that use floating point arguments to the hardfp ABI. It also

* fixes input using SDL1.2 to get sensor values from sensorfw.

*

* Known issues: Audio is muted on startup until mute is toggled.

*

* 2021-02-11 Thomas Perl <m@thp.io>

**/


#define _GNU_SOURCE


#include <stdio.h>

#include <dlfcn.h>

#include <pthread.h>

#include <SDL/SDL.h>


#define SFP __attribute__((pcs("aapcs")))


typedef unsigned int GLuint;


static void *

sensor_thread(void *user_data)

{

SDL_Init(SDL_INIT_JOYSTICK | SDL_INIT_VIDEO);


SDL_Joystick *accelerometer = SDL_JoystickOpen(0);


while (1) {

SDL_JoystickUpdate();


float x = 0.053888f * SDL_JoystickGetAxis(accelerometer, 0);

float y = 0.053888f * SDL_JoystickGetAxis(accelerometer, 1);

float z = 0.053888f * SDL_JoystickGetAxis(accelerometer, 2);


FILE *out = fopen("/dev/shm/bounce.sensor.tmp", "wb");

fprintf(out, "%f %f %f\n", -y, x, z);

fclose(out);

rename("/dev/shm/bounce.sensor.tmp", "/dev/shm/bounce.sensor");


SDL_Delay(10);

}


return NULL;

}


FILE *

fopen(const char *filename, const char *mode)

{

FILE *(*fopen_orig)(const char *, const char *) = dlsym(RTLD_NEXT, "fopen");

if (strcmp(filename, "/sys/class/i2c-adapter/i2c-3/3-001d/coord") == 0) {

static int sensor_inited = 0;

if (!sensor_inited) {

sensor_inited = 1;


pthread_t thread;

pthread_create(&thread, NULL, sensor_thread, NULL);

}


filename = "/dev/shm/bounce.sensor";

}


return fopen_orig(filename, mode);

}


#define f_f(name) float SFP name(float x) { return ((float (*)(float))dlsym(RTLD_NEXT, #name))(x); }

#define d_d(name) double SFP name(double x) { return ((double (*)(double))dlsym(RTLD_NEXT, #name))(x); }

#define f_ff(name) float SFP name(float x, float y) { return ((float (*)(float, float))dlsym(RTLD_NEXT, #name))(x, y); }

#define d_dd(name) double SFP name(double x, double y) { return ((double (*)(double, double))dlsym(RTLD_NEXT, #name))(x, y); }


f_f(sinhf) f_f(coshf) f_f(tanhf) f_f(asinf) f_f(acosf) f_f(atanf) f_f(sinf) f_f(cosf) f_f(tanf) f_f(expf) f_f(logf)

f_f(log10f) f_f(ceilf) f_f(floorf) d_d(log) d_d(sin) f_ff(atan2f) f_ff(fmodf) d_dd(atan2) d_dd(pow) d_dd(fmod)


double SFP

frexp(double value, int *exp)

{

return ((double (*)(double, int *))dlsym(RTLD_NEXT, "frexp"))(value, exp);

}


double SFP

ldexp(double x, int n)

{

return ((double (*)(double, int))dlsym(RTLD_NEXT, "ldexp"))(x, n);

}


double SFP

modf(double value, double *iptr)

{

return ((double (*)(double, double *))dlsym(RTLD_NEXT, "modf"))(value, iptr);

}


void SFP

glClearColor(float r, float g, float b, float a)

{

((void (*)(float, float, float, float))dlsym(RTLD_NEXT, "glClearColor"))(r, g, b, a);

}


void SFP

glUniform4f(GLuint location, float v0, float v1, float v2, float v3)

{

((void (*)(GLuint, float, float, float, float))dlsym(RTLD_NEXT, "glUniform4f"))(location, v0, v1, v2, v3);

}


void SFP

glUniform1f(GLuint location, float v0)

{

((void (*)(GLuint, float))dlsym(RTLD_NEXT, "glUniform1f"))(location, v0);

}


0 Add to favourites0 Bury

11 Feb 2021 7:41am GMT

04 Feb 2021

feedPlanet Maemo

Ubuntu Touch porting notes for the Redmi Note 7 Pro, part 2

This is the second part of my porting odyssey; for the first part, follow this link.

The good news is that I've done some progress; the bad news is that there are still plenty of issues, and solving them involves deep diving into nearly all components and technologies that make up the core of an Android device, so completing the porting is going to take quite some time. On the other hand, I'm learning a lot of new stuff, and I might be able to share it by writing some documentation. And, who knows, maybe work on some other device port.

Anyway, enough with the introduction! Let's see what progress I've been doing so far.

The new device tree

While asking for help to debug the audio issue I was facing (more about that later), I was also told that the lavender tree, which I was using as a reference, was obsolete. The new one was in gitlab, and was build with a totally different system, described here.

So, I picked the lavender tree and adapted it for violet: I changed the deviceinfo file to point to my kernel tree, use my kernel configuration, and use the same boot command line as before. By the way, here's my current "new" device tree. The build failed, with errors like:

In function 'memcpy',
    inlined from 'proc_ipc_auto_msgmni.part.1' at /home/mardy/projects/port/xiaomi-violet/bd/downloads/android_kernel_xiaomi_violet/ipc/ipc_sysctl.c:82:2:
/home/mardy/projects/port/xiaomi-violet/bd/downloads/android_kernel_xiaomi_violet/include/linux/string.h:340:4: error: call to '__read_overflow2' declared with attribute error: detected read beyond size of object passed as 2nd parameter
    __read_overflow2();
    ^

I then replayed the kernel build using the old system, and noticed that it was using clang as the compiler; so I changed the related flag in the deviceinfo file, and the build went past that point. It failed later, though:

/home/mardy/projects/port/xiaomi-violet/bd/downloads/android_kernel_xiaomi_violet/drivers/staging/qcacld-3.0/core/sap/src/sap_fsm.c:1498:26: error: cast to smaller integer type 'eSapStatus' from 'void *' [-Werror,-Wvoid-pointer-to-enum-cast]
                bss_complete->status = (eSapStatus) context;
                                       ^~~~~~~~~~~~~~~~~~~~

I ended up editing the Kbuild file in the module source directory, and removed the -Werror from there (as well in another place that failed a bit later). This made the build proceed until the end, where it failed because the device tree file was not found:

+ cp -v /home/mardy/projects/port/xiaomi-violet/bd/downloads/KERNEL_OBJ/arch/arm64/boot/dts/qcom/sm8150-mtp-overlay.dtbo /home/mardy/projects/port/xiaomi-violet/bd/tmp/partitions/dtbo.img
cp: cannot stat '/home/mardy/projects/port/xiaomi-violet/bd/downloads/KERNEL_OBJ/arch/arm64/boot/dts/qcom/sm8150-mtp-overlay.dtbo': No such file or directory

I quickly realized that this was due to an error of mine: the Xiaomi violet is a sm6150, not a sm8150 as mentioned in my deviceinfo file! But what overlay file should I use then, since there isn't a sm6150-mtp-overlay.dts in my source tree? Having a loot at other deviceinfo files, I saw that the deviceinfo_kernel_dtb_overlay line is not always there, so I tried commenting it out, and it helped.

The next step was getting flashable images, of course. While the device is not supported by the UBports system-image server, we can use use some scripts to create a fake OTA (Over The Air update) and generate flashable images starting from it. The steps can be read in the devel-flashable target of the .gitlab-ci.yml file (if this step is not present, we should try to find another device repository which has it. They are the following:

./build/prepare-fake-ota.sh out/device_violet.tar.xz ota
./build/system-image-from-ota.sh ota/ubuntu_command out

Once these commands have completed their execution, these commands will push the images to the device:

fastboot flash boot out/boot.img; fastboot flash system out/system.img

There's also fastboot flash recovery out/recovery.img, but I left it out since I was not interested in the recovery image at this stage. And unless you are ready to submit your port for inclusion into the "supported devices" list, I'd recommend not flashing the UT recovery image since TWRP is more powerful and will likely help you in recover your device from a broken image.

Kernel command line

It's important that the kernel command line contains the systempart parameter. In my case, the first boot failed because the command line was longer than 512 bytes, and this parameter was getting truncated. So one thing to be careful about is the length of the kernel command line.

This was fixed by removing some unnecessary kernel parameters from the deviceinfo file.

Missing thumbnails in the Gallery app, content-hub not working

Another issue I noticed is that photo thumbnails were all black in the Gallery app, and the content-hub also was not working. I noticed a relevant commit in the lavender kernel tree and found a launchpad bug which mentioned the issues I was seeing. In the Telegram channel I was told that patch is a forward port of a commit from kernel 3.4 that was present in all of the cores devices, and that it was indeed needed to have the ContentHub working.

The patch did not apply cleanly on top of my kernel tree, but luckily it was just an offset issue: adjusting the patch was easy, and indeed after applying it thumbnails started appearing in the Gallery and Imaginario could import photos again via the ContentHub.

Time and date

Time was always reset to a far away date after a reboot. This is a common issue on Qualcomm devices, and can be fixed by disabling the time service in the Android container.

https://github.com/Halium/android_device_halium_halium_arm64/pull/3

For some reason, at a certain point my override stopped working (or, more likely, the override never worked, but I happened to have fixed the issue directly modifying the vendor init file). I had to copy /android/vendor/etc/init/hw/init.qcom.rc into /usr/share/halium-overrides/, modify it and bind mount the modified file in order to get it working.

This actually seems to match my understanding of the Android init documentation, because according to the path priorities a configuration file stored under /system/ will never be able to override one stored under /vendor.

Fixing the audio configuration

Audio was not working at all. The sound indicator icon was not shown, only the raw (unstranslated) "indicator-sound" text was shown in the panel. pulseaudio was not running. Trying to run it manually (as the phablet user, since pulseaudio is run in the user session) led to this:

phablet@ubuntu-phablet:~$ pulseaudio -n -vvv -F /etc/pulse/touch-android9.pa
I: [pulseaudio] main.c: setrlimit(RLIMIT_NICE, (31, 31)) failed: Operation not permitted
I: [pulseaudio] main.c: setrlimit(RLIMIT_RTPRIO, (9, 9)) failed: Operation not permitted
...
D: [pulseaudio] cli-command.c: Parsing script '/etc/pulse/touch-android9.pa'
D: [pulseaudio] database-tdb.c: Opened TDB database '/home/phablet/.config/pulse/ubuntu-phablet-stream-volumes.tdb'
I: [pulseaudio] module-stream-restore.c: Successfully opened database file '/home/phablet/.config/pulse/ubuntu-phablet-stream-volumes'.
...
D: [pulseaudio] module.c: Checking for existence of '/usr/lib/pulse-8.0/modules/module-droid-card-28.so': success
I: [pulseaudio] module-droid-card.c: Create new droid-card
D: [pulseaudio] droid-util.c: No configuration provided for opening module with id primary
I: [pulseaudio] config-parser-xml.c: Failed to open file (/odm/etc/audio_policy_configuration.xml): No such file or directory
D: [pulseaudio] droid-config.c: Failed to parse configuration from /odm/etc/audio_policy_configuration.xml
D: [pulseaudio] config-parser-xml.c: Read /vendor/etc/audio/audio_policy_configuration.xml ...
D: [pulseaudio] config-parser-xml.c: New module: "primary"
W: [pulseaudio] config-parser-xml.c: [/vendor/etc/audio/audio_policy_configuration.xml:78] Could not find element attribute "samplingRates"
E: [pulseaudio] config-parser-xml.c: [/vendor/etc/audio/audio_policy_configuration.xml:78] Failed to parse element <profile>
E: [pulseaudio] config-parser-xml.c: parsing aborted at line 78
D: [pulseaudio] droid-config.c: Failed to parse configuration from /vendor/etc/audio/audio_policy_configuration.xml
D: [pulseaudio] config-parser-xml.c: Read /vendor/etc/audio_policy_configuration.xml ...
D: [pulseaudio] config-parser-xml.c: New module: "primary"
W: [pulseaudio] config-parser-xml.c: [/vendor/etc/audio_policy_configuration.xml:78] Could not find element attribute "samplingRates"
E: [pulseaudio] config-parser-xml.c: [/vendor/etc/audio_policy_configuration.xml:78] Failed to parse element <profile>
E: [pulseaudio] config-parser-xml.c: parsing aborted at line 78
D: [pulseaudio] droid-config.c: Failed to parse configuration from /vendor/etc/audio_policy_configuration.xml
I: [pulseaudio] config-parser-legacy.c: Failed to open config file (/vendor/etc/audio_policy.conf): No such file or directory
D: [pulseaudio] droid-config.c: Failed to parse configuration from /vendor/etc/audio_policy.conf
I: [pulseaudio] config-parser-xml.c: Failed to open file (/system/etc/audio_policy_configuration.xml): No such file or directory
D: [pulseaudio] droid-config.c: Failed to parse configuration from /system/etc/audio_policy_configuration.xml
I: [pulseaudio] config-parser-legacy.c: Failed to open config file (/system/etc/audio_policy.conf): No such file or directory
D: [pulseaudio] droid-config.c: Failed to parse configuration from /system/etc/audio_policy.conf
E: [pulseaudio] droid-config.c: Failed to parse any configuration.
...

Indeed, line 78 of /vendor/etc/audio_policy_configuration.xml had an error, where a property was spelt like simplingRate instead of samplingRate. However, the "vendor" partition is read-only, so I couldn't change that file directly. Another option could have been creating a fixed copy of the file and place it with /system/etc/audio_policy_configuration.xml, but the "system" is also read-only (there are ways to modify these partitions, of course, but I couldn't find a clean way to do it from the device tree scripts. So I went for the bind-mount approach: I would ship the fixed file in some other directory of the file-system, and then modify the /etc/init/mount-android.conf file (this is the job that upstart executes before starting the Android LXC container) to bind-mount the file onto /vendor/etc/audio_policy_configuration.xml.

This worked, but my joy was short-lived: audio was coming up only once every 5 boots or so. I will not list here all the things I tried, as they were plenty of them; and more than once I went to sleep happy and convinced of having fixed the issue for good, until the next day the device booted without audio. It was clearly a timing issue occurring in the early boot, because one thing I clearly noticed very early on is that in those cases when the audio was booting, the following lines appeared in the kernel log:

[    7.130057] wcd937x_codec wcd937x-codec: msm_cdc_dt_parse_vreg_info: cdc-vdd-ldo-rxtx: vol=[1800000 1800000]uV, curr=[25000]uA, ond 0
[    7.130068] wcd937x_codec wcd937x-codec: msm_cdc_dt_parse_vreg_info: cdc-vddpx-1: vol=[1800000 1800000]uV, curr=[10000]uA, ond 0
[    7.130076] wcd937x_codec wcd937x-codec: msm_cdc_dt_parse_vreg_info: cdc-vdd-mic-bias: vol=[3296000 3296000]uV, curr=[25000]uA, ond 0
[    7.130084] wcd937x_codec wcd937x-codec: msm_cdc_dt_parse_vreg_info: cdc-vdd-buck: vol=[1800000 1800000]uV, curr=[650000]uA, ond 1
[    7.137759] wcd937x_codec wcd937x-codec: bound wcd937x-slave.1170224 (ops cleanup_module [wcd937x_slave_dlkm])
[    7.138065] wcd937x_codec wcd937x-codec: bound wcd937x-slave.1170223 (ops cleanup_module [wcd937x_slave_dlkm])

I started adding some printk to the kernel driver, and modified it slightly to register itself with the module_driver() macro instead of the simpler, but logless, module_platform_driver(). This showed that the driver was always loaded at about 7 seconds, but the platform_driver_register() method only called the driver's bind method (wcd937x_bind()) in those boots where audio was working.

After more debugging into platform_driver_register(), I stumbled upon the platform_match() function, added some debugging message in there to print the device name and the driver name, and observed how in those boots where audio was failing this function was called to find a driver for the wcd937x_codec device before the wcd937x_codec driver (provided by the wcd937x_dlmk module) was available. So, I tried adding wcd937x_dlmk to /etc/modules-load.d/modules.conf and this caused the driver to be loaded at about 3 seconds, and apparently fixed the audio issue. At least, till the time of writing this, I never had my phone boot without audio anymore.

Not all is fine with the audio, unfortunately: the mic records a background noise along with the actual sounds, and the recording volume is quite low. This also affects the call quality. On the other hand, the noise disappears when recording happens via the earphones. But I've yet to investigate this; I hope to give you some updates in part three.

0 Add to favourites0 Bury

04 Feb 2021 8:34pm GMT

17 Jan 2021

feedPlanet Maemo

Ubuntu Touch porting notes for the Redmi Note 7 Pro

In case you have a sense of deja-vu when reading this post, it's because indeed this is not the first time I try porting a device to Ubuntu Touch. The previous attempt, however, was with another phone model (and manufacturer), and did not have a happy ending. This time it went better, although the real ending is still far away; but at least I have something to celebrate.

The phone

I made myself a Christmas present and bought a Xiaomi Redmi Note 7 Pro, a dual-SIM phone from 2019 with 6GB of RAM and 128GB of flash storage. To be totally honest, I bought it by mistake: the phone I really wanted to buy is the Redmi Note 7 (without the "Pro"), because it's a modern phone that is working reasonable well with Ubuntu Touch. The online shop where I bought it from let me choose some options, including the RAM size, so I chose the maximum available (6GB) without being aware that this would mean that I would be buying the "Pro" device - the shop did not alter the item name, so I couldn't really know. Unfortunately, the two versions are two rather different beasts, powered by different SoC; both are produced by Qualcomm, so they are not that different, but it's enough to make the installation of Ubuntu Touch impossible.

But between the choice of retuning the phone to the shop and begin a new porting adventure, I stood firm and went for the latter. Hopefully I won't regret it (if everything goes bad, I can still use it with LineageOS, which runs perfectly on it).

Moreover, there already exist a port of Ubuntu Touch for this phone, which actually works reasonably well (I tried it briefly, and many things were working), but the author claims to be a novice and indeed did not follow the best git practices when working on the source code, so it's hard to understand what was changed and why. But if you are looking for a quick way to get Ubuntu Touch working on this phone, you are welcome to have a look at this Telegram channel.

What follows are the raw notes of my attempts. They are here so that search engines can index the error messages and the various logs, and hopefully help someone hitting similar errors on other devices to find his way out.

Getting Halium and the device source code

Installed the dependencies like in the first step. repo was not found in the Ubuntu 20.04 archives, but I had it installed anyway due to my work on a Yocto device.

Since my device has Android 9 on it, I went for Halium 9:

repo init -u git://github.com/Halium/android.git -b halium-9.0 --depth=1
repo sync -c -j 16

The official LineageOS repository for the Xiaomi Redmi Note 7 Pro is android_device_xiaomi_violet, but the page with the official build has been taken down (some DMCA violation, if you believe the forums) and no development has been happening since last year. A more active forum thread uses another repository which seems to be receiving more frequent updates, so I chose to base my port on that.

I actually tested that LineageOS image on my phone, and verified that all the hardware was working properly.

Initially, I created forks of the relevant repository under my own gitlab account, but then I though (especially looking at the Note 7 port) that creating a group just for this port would make people's life easier, because they wouldn't need to navigate through my 1000 personal projects to find what is relevant for this port. So, I created these forks:

Next, created the halium/devices/manifests/xiaomi_violet.xml file with this content:

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
    <!-- Remotes -->
    <remote name="ubuntu-touch-xiaomi-violet"
            fetch="https://gitlab.com/ubuntu-touch-xiaomi-violet"
            revision="halium-9.0"/>

    <!-- Device Tree -->
    <project path="device/xiaomi/violet"
             name="android_device_xiaomi_violet"
             remote="ubuntu-touch-xiaomi-violet" />

    <!-- Kernel -->
    <project path="kernel/xiaomi/violet"
             name="android_kernel_xiaomi_violet"
             remote="ubuntu-touch-xiaomi-violet" />

    <!-- Proprietary/Vendor blobs -->
    <project path="vendor/xiaomi/violet"
             name="android_vendor_xiaomi_violet"
             remote="ubuntu-touch-xiaomi-violet" />
</manifest>

Fetching all the sources mentioned in the manifest:

./halium/devices/setup violet

Full output:

*****************************************
I: Configuring for device xiaomi_violet
*****************************************
Fetching projects: 100% (393/393), done.
hardware/qcom/audio-caf/apq8084: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning.
hardware/qcom/audio-caf/msm8916: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning.
hardware/qcom/audio-caf/msm8952: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning.
hardware/qcom/audio-caf/msm8960: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning.
hardware/qcom/audio-caf/msm8974: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning.
hardware/qcom/audio-caf/msm8994: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning.
hardware/qcom/audio-caf/msm8996: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning.
hardware/qcom/audio-caf/msm8998: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning.
hardware/qcom/audio-caf/sdm845: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning.
hardware/qcom/audio-caf/sm8150: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning.
hardware/qcom/audio/default: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning.
hardware/qcom/display: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning.
hardware/qcom/display-caf/apq8084: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning.
hardware/qcom/display-caf/msm8916: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning.
hardware/qcom/display-caf/msm8952: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning.
hardware/qcom/display-caf/msm8960: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning.
hardware/qcom/display-caf/msm8974: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning.
hardware/qcom/display-caf/msm8994: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning.
hardware/qcom/display-caf/msm8996: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning.
hardware/qcom/display-caf/msm8998: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning.
hardware/qcom/display-caf/sdm845: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning.
hardware/qcom/display-caf/sm8150: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning.
hardware/qcom/media: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning.
hardware/qcom/media-caf/apq8084: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning.
hardware/qcom/media-caf/msm8916: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning.
hardware/qcom/media-caf/msm8952: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning.
hardware/qcom/media-caf/msm8960: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning.
hardware/qcom/media-caf/msm8974: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning.
hardware/qcom/media-caf/msm8994: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning.
hardware/qcom/media-caf/msm8996: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning.
hardware/qcom/media-caf/msm8998: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning.
hardware/qcom/media-caf/sdm845: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning.
hardware/qcom/media-caf/sm8150: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning.
hardware/ril: Shared project LineageOS/android_hardware_ril found, disabling pruning.
hardware/ril-caf: Shared project LineageOS/android_hardware_ril found, disabling pruning.
hardware/qcom/wlan: Shared project LineageOS/android_hardware_qcom_wlan found, disabling pruning.
hardware/qcom/wlan-caf: Shared project LineageOS/android_hardware_qcom_wlan found, disabling pruning.
hardware/qcom/bt: Shared project LineageOS/android_hardware_qcom_bt found, disabling pruning.
hardware/qcom/bt-caf: Shared project LineageOS/android_hardware_qcom_bt found, disabling pruning.
Updating files: 100% (67031/67031), done.nel/testsUpdating files:  36% (24663/67031)
lineage/scripts/: discarding 1 commits
Updating files: 100% (1406/1406), done.ineageOS/android_vendor_qcom_opensource_thermal-engineUpdating files:  47% (666/1406)
Checking out projects: 100% (392/392), done.
*******************************************
I: Refreshing device vendor repository: device/xiaomi/violet
I: Processing proprietary blob file: device/xiaomi/violet/./proprietary-files.txt
I: Processing fstab file: device/xiaomi/violet/./rootdir/etc/fstab.qcom
I: Removing components relying on SettingsLib from: device/xiaomi/violet
*******************************************

Starting the build

Setting up the environment:

$ source build/envsetup.sh
including device/xiaomi/violet/vendorsetup.sh
including vendor/lineage/vendorsetup.sh
$

Running the breakfast command:

$ breakfast violet
including vendor/lineage/vendorsetup.sh
Trying dependencies-only mode on a non-existing device tree?

============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=9
LINEAGE_VERSION=16.0-20210109-UNOFFICIAL-violet
TARGET_PRODUCT=lineage_violet
TARGET_BUILD_VARIANT=userdebug
TARGET_BUILD_TYPE=release
TARGET_ARCH=arm64
TARGET_ARCH_VARIANT=armv8-a
TARGET_CPU_VARIANT=kryo300
TARGET_2ND_ARCH=arm
TARGET_2ND_ARCH_VARIANT=armv8-a
TARGET_2ND_CPU_VARIANT=cortex-a75
HOST_ARCH=x86_64
HOST_2ND_ARCH=x86
HOST_OS=linux
HOST_OS_EXTRA=Linux-5.4.0-58-generic-x86_64-Ubuntu-20.04.1-LTS
HOST_CROSS_OS=windows
HOST_CROSS_ARCH=x86
HOST_CROSS_2ND_ARCH=x86_64
HOST_BUILD_TYPE=release
BUILD_ID=PQ3A.190801.002
OUT_DIR=/home/mardy/projects/port/halium/out
PRODUCT_SOONG_NAMESPACES= hardware/qcom/audio-caf/sm8150 hardware/qcom/display-caf/sm8150 hardware/qcom/media-caf/sm8150
============================================

Building the kernel

The configuration needs to be adapted for Halium. Locating the kernel config:

$ grep "TARGET_KERNEL_CONFIG" device/xiaomi/violet/BoardConfig.mk 
TARGET_KERNEL_CONFIG := vendor/violet-perf_defconfig
$

Getting the Mer checker tool and running it:

git clone https://github.com/mer-hybris/mer-kernel-check
cd mer-kernel-check
./mer_verify_kernel_config ../kernel/xiaomi/violet/arch/arm64/configs/vendor/violet-perf_defconfig

Prints lots of warnings, starting with:

WARNING: kernel version missing, some reports maybe misleading

To help the tool, we need to let him know the kernel version. It can be seen at the beginning of the kernel Makefile, located in ../kernel/xiaomi/violet/Makefile; in my case it was

VERSION = 4
PATCHLEVEL = 14
SUBLEVEL = 83
EXTRAVERSION =

So I edited the configuration file ../kernel/xiaomi/violet/arch/arm64/configs/vendor/violet-perf_defconfig and added this line at the beginning:

# Version 4.14.83

then ran the checker tool again. This time the output was a long list of kernel options that needed to be fixed, but as I went asking for some explanation in the Telegram channel for Ubuntu Touch porters, I was told that I could/should skip this test and instead use the check-kernel-config tool from Halium. I downloaded it, made it executable (chmod +x check-kernel-config) and ran it:

./check-kernel-config kernel/xiaomi/violet/arch/arm64/configs/vendor/violet-perf_defconfig
[...lots of red and green lines...]
Config file checked, found 288 errors that I did not fix.

I ran it again with the -w option, and it reportedly fixed 287 errors. Weird, does that mean that an error was still left? I ran the tool again (without -w), and it reported 2 errors. Ran it once more in write mode, and if fixed them. So, one might need to run it twice.

Next, added the line

BOARD_KERNEL_CMDLINE += console=tty0

in device/xiaomi/violet/BoardConfig.mk. One more thing that needs to be done before starting the build is fixing the mount points, so I opened device/xiaomi/violet/rootdir/etc/fstab.qcom and changed the type of the userdata partition from f2fs to ext4. There were no lines with the context option, so that was the only change I needed to do.

At this point, while browsing throught the documentation, I found a link to this page which contains some notes on Halium 9 porting, which are substantially different from the official porting guide in halium.org (which I was already told in Telegram to be obsolete in several points).

So, following the instructions from this new link I ran

hybris-patches/apply-patches.sh --mb

which completed successfully.

Then, continuing following the points from this page, I edited device/xiaomi/violet/lineage_violet.mk, commented out the lines

$(call inherit-product, $(SRC_TARGET_DIR)/product/full_base_telephony.mk)
# ...
$(call inherit-product, vendor/lineage/config/common_full_phone.mk)

and replaced the first one with a similar line pointing to halium.mk. For the record, a find revealed that the SRC_TARGET_DIR variable in my case was build/make/target/. It contained also the halium.mk file, which was created by the hybris patches before. As for removing the Java dependencies, I cound't find any modules similar to those listed in this commit in any of the makefiles in my source tree, so I just started the build:

source build/envsetup.sh && breakfast violet
make halium-boot

This failed the first time with the compiler being killed (out of memory, most likely), but it succeeded on the second run. There was a suspicious warning, though:

drivers/input/touchscreen/Kconfig:1290:warning: multi-line strings not supported

And indeed my kernel/xiaomi/violet/drivers/input/touchscreen/Kconfig had this line:

source "drivers/input/touchscreen/ft8719_touch_f7b/Kconfig

(notice the unterminated string quote). I don't know if this had any impact on the build, but just to be on the safe side I added the missing quote and rebuilt.

Building the system image

Running

make systemimage

failed pretty soon:

ninja: error: '/home/mardy/projects/port/halium/out/soong/host/linux-x86/framework/turbine.jar', needed by '/home/mardy/projects/port/halium/out/soong/.intermediates/libcore/core-oj/android_common/turbine/core-oj.jar', missing and no known rule to make it

The error is due to all the Java-related stuff that I should have disabled but couldn't find. So, I tried to have a look at the changes made on another xiaomi device (lavender, the Redmi note 7, which might not be that different, I thought) and started editing device/xiaomi/violet/device.mk and removing a couple of Android packages. Eventually the build proceeded, just to stop at a python error:

  File "build/make/tools/check_radio_versions.py", line 56
    print "*** Error opening \"%s.sha1\"; can't verify %s" % (fn, key)
          ^
SyntaxError: invalid syntax

Yes, it's the python3 vs python2 issue, since in my system python is python version 3. In order to fix it, I created a virtual environment:

virtualenv --python 2.7 ../python27  # adjust the path to your prefs
source ../python27/bin/activate

Remember that the second line must be run every time you'll need to setup the Halium build environment (that is, every time you run breakfast).

The build then proceeded for several minutes, until it failed due to some unresolved symbols:

prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/aarch64-linux-android/bin/ld.gold: error: /home/mardy/projects/port/halium/out/target/product/violet/obj/STATIC_LIBRARIES/lib_driver_cmd_qcwcn_intermediates/lib_driver_cmd_qcwcn.a: member at 7694 is not an ELF object
external/wpa_supplicant_8/hostapd/src/drivers/driver_nl80211.c:7936: error: undefined reference to 'wpa_driver_set_p2p_ps'
/home/mardy/projects/port/halium/out/target/product/violet/obj/EXECUTABLES/hostapd_intermediates/src/drivers/driver_nl80211.o:driver_nl80211.c:wpa_driver_nl80211_ops: error: undefined reference to 'wpa_driver_set_ap_wps_p2p_ie'
/home/mardy/projects/port/halium/out/target/product/violet/obj/EXECUTABLES/hostapd_intermediates/src/drivers/driver_nl80211.o:driver_nl80211.c:wpa_driver_nl80211_ops: error: undefined reference to 'wpa_driver_get_p2p_noa'
/home/mardy/projects/port/halium/out/target/product/violet/obj/EXECUTABLES/hostapd_intermediates/src/drivers/driver_nl80211.o:driver_nl80211.c:wpa_driver_nl80211_ops: error: undefined reference to 'wpa_driver_set_p2p_noa'
/home/mardy/projects/port/halium/out/target/product/violet/obj/EXECUTABLES/hostapd_intermediates/src/drivers/driver_nl80211.o:driver_nl80211.c:wpa_driver_nl80211_ops: error: undefined reference to 'wpa_driver_nl80211_driver_cmd'

As people told me in the Telegram channel, this driver is not used since "our wpa_supplicant talks to kernel directly". OK, so I simply disabled the driver, copying again from the lavender Halium changes: commented out the BOARD_WLAN_DEVICE := qcwcn line (and all lines referring this variable) from device/xiaomi/violet/BoardConfig.mk and ran make systemimage again. This time, surprisingly, it all worked.

Try it out

I first tried to flash the kernel only. I rebooted into fastboot, and on my PC ran this:

cout
fastboot flash boot halium-boot.img

I then rebooted my phone, but after showing the boot logo for a few seconds it would jump to the fastboot screen. Indeed, flashing the previous kernel would restore the normal boot, so there had to be something wrong with my own kernel.

While looking at what the problem could be, I noticed that in the lavender port the author did not modify the lineageos kernel config file in place, but instead created a new one and changed the BoardConfig.mk file to point to his new copy. Since it sounded like a good idea, I did the same and created kernel/xiaomi/violet/arch/arm64/configs/vendor/violet_halium_defconfig for the Halium changes. And then the line in the board config file became

TARGET_KERNEL_CONFIG := vendor/violet_halium_defconfig

I then continued investigating the boot issue, and I was told that it might have been due to an Android option, skip_initramfs, which is set by the bootloader and causes our Halium boot to fail. The fix is to just disable this option in the kernel, by editing init/initramfs.c and change the skip_initramfs_param function to always set the do_skip_initramfs variable to 0, rather than to 1. After doing this, the boot proceeded to show the Ubuntu splash screen with the five dots being lit, but it didn't proceed from there.

Setting up the udev rules

Even in this state, the device was detected by my host PC and these lines appeared in the system log:

kernel: usb 1-3: new high-speed USB device number 26 using xhci_hcd
kernel: usb 1-3: New USB device found, idVendor=0fce, idProduct=7169, bcdDevice= 4.14
kernel: usb 1-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
kernel: usb 1-3: Product: Unknown
kernel: usb 1-3: Manufacturer: GNU/Linux Device
kernel: usb 1-3: SerialNumber: GNU/Linux Device on usb0 10.15.19.82

Indeed, I guess the reason I could do this without even flashing my systemimage is because I had first flashed another UT system image from another porter. I'm not sure if I'd had the same results with my own image. Anyway, the USB networking was there, so I connected and ran the following commands to generate the udev rules:

ssh phablet@10.15.19.82
# used 0000 as password
sudo -i
# same password again
cd /home/phablet
DEVICE=violet
cat /var/lib/lxc/android/rootfs/ueventd*.rc /vendor/ueventd*.rc \
        | grep ^/dev \
        | sed -e 's/^\/dev\///' \
        | awk '{printf "ACTION==\"add\", KERNEL==\"%s\", OWNER=\"%s\", GROUP=\"%s\", MODE=\"%s\"\n",$1,$3,$4,$2}' \
        | sed -e 's/\r//' \
        >70-$DEVICE.rules

I then copied (with scp) this file to my host PC, and I moved it to device/xiaomi/violet/ubuntu/70-violet.rules (I had to create the ubuntu directory first). Then I edited the device/xiaomi/violet/device.mk file and added these lines at the end:

### Ubuntu Touch ###
PRODUCT_COPY_FILES += \
    $(LOCAL_PATH)/ubuntu/70-violet.rules:system/halium/lib/udev/rules.d/70-android.rules
### End Ubuntu Touch ###

It was now time to try my own system image. I rebooted my device into TWRP, I cloned the halium-install repository into my halium build dir, downloaded a rootfs for Halium9, and ran

./halium-install/halium-install -p ut \
    ~/Downloads/ubuntu-touch-android9-arm64.tar.gz \
    out/target/product/violet/system.img

The first time this failed because the simg2img tool was not installed. The second time it proceeded to create the image, asked me for a password for the phablet user (gave "0000") and pushed the image onto the device, into the /data/ partition. I then rebooted.

My first Ubuntu Touch image

Upon reboot, the usual splash screen appeared, followed by several seconds of black screen. It definitely didn't look right, but at least it proved that something had been flashed. After some more seconds, to my big surprise, the Ubuntu boot screen appeared, just with a smaller logo than how it used to be before, which also confimed that my system image was being used - in fact, I did not adjust the GRID_UNIT_PX variable before. Since this was an easy fix, I chose to focus on that, rather than fix the boot issues (indeed, my device did not move on from the Ubuntu boot screen). SSH was working.

I took the scaling.conf file from the lavender changes, put it in device/xiaomi/violet/ubuntu/ and added this line in the PRODUCT_COPY_FILES in device.mk:

$(LOCAL_PATH)/ubuntu/scaling.conf:system/halium/etc/ubuntu-touch-session.d/android.conf

I initially used system/ubuntu/etc/... as the target destination for the config file, like in the lavender commit, but this didn't work out for me. Then I changed the path to start with system/halium/, like it's mentioned in the ubports documentation, but it apparently had no effect either.

After a couple of days spent trying to understand why my files were not appearing under /etc, it turned out that the device was not using my system image at all: with halium-install I had my image installed in /userdata/system.img, while the correct path for Halium 9 devices is /userdata/android-rootfs.img. I was told that the option -s of halium-install would do the trick. Instead of re-running the script, I took the shortcut of renaming /userdata/system.img to /userdata/android-rootfs.img and after rebooting I could see that the Ubuntu logo was displayed at the correct size. And indeed my system image was being used.

So I started to debug why unity8 didn't start. The logs terminated with this line:

terminate called after throwing an instance of 'std::runtime_error'
  what():  org.freedesktop.DBus.Error.NoReply: Message recipient disconnected from message bus without replying
initctl: Event failed

It means, that unity8 did not handle correctly the situation where another D-Bus service crashed while handing a call from unity8. The problem now was how to figure out which service it was. I ran dbus-monitor and restarted unity8 (initctl start unity8), then examined the dbus logs; I saw the point where unity8 got disconnected from the bus, but before that point I didn't find any failed D-Bus calls. So it had to be the system bus. I did exactly the same steps, just this time after running dbus-monitor --system as root, I found the place where unity8 got disconnected, and found this D-bus error shortly before that:

method call time=1605676421.968667 sender=:1.306 -> destination=com.ubuntu.biometryd.Service serial=3 path=/default_device; interface=com.ubuntu.biometryd.Device; member=Identifier
method return time=1605676421.970222 sender=:1.283 -> destination=:1.306 serial=4 reply_serial=3
   object path "/default_device/identifier"
...
method call time=1605676421.974218 sender=:1.283 -> destination=org.freedesktop.DBus serial=6 path=/org/freedesktop/DBus; interface=org.freedesktop.DBus; member=GetConnectionAppArmorSecurityContext
   string ":1.306"
error time=1605676421.974278 sender=org.freedesktop.DBus -> destination=:1.283 error_name=org.freedesktop.DBus.Error.AppArmorSecurityContextUnknown reply_serial=6
   string "Could not determine security context for ':1.306'"
...
signal time=1605676421.989074 sender=org.freedesktop.DBus -> destination=:1.283 serial=6 path=/org/freedesktop/DBus; interface=org.freedesktop.DBus; member=NameLost
   string "com.ubuntu.biometryd.Service"
...
error time=1605676421.989302 sender=org.freedesktop.DBus -> destination=:1.306 error_name=org.freedesktop.DBus.Error.NoReply reply_serial=4
   string "Message recipient disconnected from message bus without replying"

So, it looks like unity8 (:1.306) made a request to biometryd, who asked the D-Bus daemon what was the AppArmor label of the caller, but since I didn't backport the D-Bus mediation patches for AppArmor, the label could not be resolved. biometryd decided to crash instead of properly handling the error, and so did unity8.

So I backported the AppArmor patches. I took them from the Canonical kernel repo, but they did not apply cleanly because they are meant to be applied on top of a pristine 4.14 branch, whereas the Android kernel had already some AppArmor security fixes backported from later releases. So I set and quickly inspected the contents of each patch, and found out that a couple of them had already been applied, and the last patch had to be applied as the first (yeah, it's hard to explain, but it all depends on other patches that Android has backported from newer kernels). Anyway, after rebuilding the kernel and reflashing it, my phone could finally boot to unity8!

Conclusion (of the first episode)

The actual porting, as I've been told, starts here. What has been documented here are only the very first steps of the bring-up; what awaits me now is to make all the hardware subsystems work properly, and this, according to people more experienced in porting, is the harder part.

So far, very few things work, to the point that it's faster to me to list the things that do work; it's safe to assume that all what is not listed here is not working:

  • Mir, with graphics and input: Unity8 starts and is usable
  • Camera: can take photos; video recording start but an error appears when the stop button is pressed
  • Flashlight
  • Fingerprint reader: surprisingly, this worked out of the box
  • Screen brightness (though it can be changed only manually)

For all the rest, please keep an eye on this blog: I'll write, when I make some progress!

0 Add to favourites0 Bury

17 Jan 2021 7:15pm GMT

17 Dec 2020

feedPlanet Maemo

Avast, Qt6 announcing new QPromise and QFuture APIs

Qt published its New_Features in Qt 6.0.

Some noteworthy items in their list:

I like to think I had my pirate-hook in it at least a little bit with QTBUG-61928.

I need to print this out and put it above my bed:
Thiago Macieira added a comment - 13 Jul '17 03:51
You're right
Philip Van Hoof added a comment - 13 Jul '17 07:32
Damn, and I was worried the entire morning that I had been ranting again.
Thiago Macieira added a comment - 13 Jul '17 16:06
oh, you were ranting. Doesn't mean you're wrong.
Thanks for prioritizing this Thiago.

0 Add to favourites0 Bury

17 Dec 2020 9:29pm GMT

06 Dec 2020

feedPlanet Maemo

How to generate random points on a sphere

This question often pops up, when you need a random direction vector to place things in 3D or you want to do a particle simulation.

We recall that a 3D unit-sphere (and hence a direction) is parametrized only by two variables; elevation \theta \in [0; \pi] and azimuth \varphi \in [0; 2\,\pi] which can be converted to Cartesian coordinates as

\begin{aligned} x &= \sin\theta \, \cos\varphi \\ y &= \sin\theta \, \sin\varphi \\ z &= \cos\theta \end{aligned}

If one takes the easy way and uniformly samples this parametrization in numpy like

phi = np.random.rand() * 2 * np.pi
theta = np.random.rand() * np.pi

One (i.e. you as you are reading this) ends with something like this:

While the 2D surface of polar coordinates uniformly sampled (left), we observe a bias of sampling density towards the poles when projecting to the Cartesian coordinates (right).
The issue is that the cos mapping of the elevation has an uneven step size in Cartesian space, as you can easily verify: cos^{'}(x) = sin(x).

The solution is to simply sample the elevation in the Cartesian space instead of the spherical space - i.e. sampling z \in [-1; 1]. From that we can get back to our elevation as \theta = \arccos z:

z = 1 - np.random.rand() * 2 # convert rand() range 0..1 to -1..1
theta = np.arccos(z)

As desired, this compensates the spherical coordinates such that we end up with uniform sampling in the Cartesian space:

Custom opening angle

If you want to further restrict the opening angle instead of sampling the full sphere you can also easily extend the above. Here, you must re-map the cos values from [1; -1] to [0; 2] as

cart_range = -np.cos(angle) + 1 # maximal range in cartesian coords
z = 1 - np.random.rand() * cart_range
theta = np.arccos(z)

Optimized computation

If you do not actually need the parameters \theta, \varphi, you can spare some trigonometric functions by using \sin \theta = \sqrt { 1 - z^2} as

\begin{aligned} x &= \sqrt { 1 - z^2} \, \cos\varphi \\ y &= \sqrt { 1 - z^2} \, \sin\varphi \end{aligned} 0 Add to favourites0 Bury

06 Dec 2020 1:12am GMT

18 Nov 2020

feedPlanet Maemo

Self-built NAS for Nextcloud hosting

With Google cutting its unlimited storage and ending the Play Music service, I decided to use my own Nextcloud more seriously.
In part because Google forced all its competitors out of the market, but mostly because I want to be independent of any cloudy services.

The main drawback of my existing Nextcloud setup, that I have written about here, was missing redundancy; the nice thing about putting your stuff in the cloud is that you do not notice if one of the storage devices fails - Google will take care of providing you with a backup copy of your data.

Unfortunately, the Intel NUC based build I used, while offering great power efficiency did not support adding a second HDD to create a fail-safe RAID1 setup. Therefore I had to upgrade.

As I still wanted to keep things power-efficient in a small form-factor, my choice fell on the ASRock DeskMini series. Here, I went with the AMD Variant (A300) in order to avoid paying the toll of spectre mitigations with Intel (resulting in just 80% of baseline performance).

Size comparison between the A300 and a Gigabyte BRIX

The photos above show you the size difference, which is considerable - yet necessary to cram two 2.5″ SATA drives next to each other.
Here, keep in mind that while the NUC devices have their CPU soldered on, we are getting the standardized STX form-factor with the A300, which means you can replace and upgrade the mainboard and the CPU as you wish, while with a NUC you are basically stuck with what you bought initially.

The full config of the build is as follows and totals at about 270€

Note, that I deliberately chose SSDs from different vendors to reduce the risk of simultaneous failure.
Also, while the 3000G is not the fastest AMD CPU, it is sufficient to host nextcloud and is still a nice upgrade from the Intel Celeron I used previously.
Furthermore, its 35W TDP nicely fits the constrained cooling options. Note, that you can limit for Ryzen 3/5 CPUs to 35W in the BIOS as well, so there is not need to get their GE variants.
However, for a private server you probably do not need that CPU power anyway, so just go with the Athlon 3000G for half the price.

Unfortunately, the A300 system is not designed for passive cooling and comes with a quite annoying CPU fan. To me the fan coming with the Athlon 3000G was less annoying, so I used that instead.
Anyway, you should set the fan RPM to 0% below 50° C in the BIOS, which results in 800 RPM and is unhearable while keeping the CPU reasonably cool.

Power Consumption

As the machine will run 24/7, power consumption is an important factor. As a rule of thumb you can calculate with 1€ for 1 Watt drained non-stop for 1 Year.

The 35W TDP gives us a upper limit of what the system will consume on persistent load - however the more interesting measure is the idle consuption as thats the state the system will be most of the time.

As I already tried some builds with different architectures, we have some interesting values to compare to, putting the A300 build in perspective

Build CPU Idle Load
Odroid U3 Exynos4412 3.7 W 9 W
Gigabyte BRIX Intel N3350 4.5 W 9.6 W
A300 Athlon 3000G 7.3 W 33.6 W

While you can obviously push the system towards 35W by with multiple simultaneous users, the 7.3 W idle consumption is quite nice.
Keep in mind, that the A300 was measured with two SATA drives operating as RAID1. If you only use one you can subtract 1W - at which point it is only 1.5 W away from the considerably weaker NUC system.

You might now wonder, whether the load or the idle measure is closer to the typical consumption. For this I measured the consumption for one day, which totalled at 0.18 kWh - or 7.5 Watts.

Power optimizations

To reach that 7.3 W idle, you need to tune some settings though. The most important one and luckily the easiest to fix is using a recent kernel.
If you are on Ubuntu 18.04, update to 20.04 or install the hwe kernel (5.4.0) - it saves you 4 Watts (11.3 to 7.3).

For saving about 0.5 watts, you can downgrade the network interface from 1Gbit to 100Mbit by executing

ethtool -s enp2s0 speed 100 duplex full autoneg on

Additionally, you can use Intels powertop to tune your system settings for power saving as

powertop --auto-tune

0 Add to favourites0 Bury

18 Nov 2020 8:16pm GMT

12 Nov 2020

feedPlanet Maemo

Imaginario 0.10

Today I released Imaginario 0.10. No bigger changes there, but two important bugfixes.

0 Add to favourites0 Bury

12 Nov 2020 4:12pm GMT

16 Oct 2020

feedPlanet Maemo

Figuring out corrupt stacktraces on ARM

If you're developing C/C++ on embedded devices, you might already have stumbled upon a corrupt stacktrace like this when trying to debug with gdb:

(gdb) bt
#0  0xb38e32c4 in pthread_getname_np () from /home/enrique/buildroot/output5/staging/lib/libpthread.so.0
#1  0xb38e103c in __lll_timedlock_wait () from /home/enrique/buildroot/output5/staging/lib/libpthread.so.0
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

In these cases I usually give up gdb and try to solve my problems by adding printf()s and resorting to other tools. However, there are times when you really really need to know what is in that cursed stack.

ARM devices subroutine calls work by setting the return address in the Link Register (LR), so the subroutine knows where to point the Program Counter (PC) register to. While not jumping into subroutines, the values of the LR register is saved in the stack (to be restored later, right before the current subroutine returns to the caller) and the register can be used for other tasks (LR is a "scratch register"). This means that the functions in the backtrace are actually there, in the stack, in the form of older saved LRs, waiting for us to get them.

So, the first step would be to dump the memory contents of the backtrace, starting from the address pointed by the Stack Pointer (SP). Let's print the first 256 32-bit words and save them as a file from gdb:

(gdb) set logging overwrite on
(gdb) set logging file /tmp/bt.txt
(gdb) set logging on
Copying output to /tmp/bt.txt.
(gdb) x/256wa $sp
0xbe9772b0:     0x821e  0xb38e103d   0x1aef48   0xb1973df0
0xbe9772c0:      0x73d  0xb38dc51f        0x0          0x1
0xbe9772d0:   0x191d58    0x191da4   0x19f200   0xb31ae5ed
...
0xbe977560: 0xb28c6000  0xbe9776b4        0x5      0x10871 <main(int, char**)>
0xbe977570: 0xb6f93000  0xaaaaaaab 0xaf85fd4a   0xa36dbc17
0xbe977580:      0x130         0x0    0x109b9 <__libc_csu_init> 0x0
...
0xbe977690:        0x0         0x0    0x108cd <_start>  0x0
0xbe9776a0:        0x0     0x108ed <_start+32>  0x10a19 <__libc_csu_fini> 0xb6f76969
(gdb) set logging off
Done logging to /tmp/bt.txt.

Gdb already can name some of the functions (like main()), but not all of them. At least not the ones more interesting for our purpose. We'll have to look for them by hand.

We first get the memory page mapping from the process (WebKit's WebProcess in my case) looking in /proc/pid/maps. I'm retrieving it from the device (named metro) via ssh and saving it to a local file. I'm only interested in the code pages, those with executable ('x') permissions:

$ ssh metro 'cat /proc/$(ps axu | grep WebProcess | grep -v grep | { read _ P _ ; echo $P ; })/maps | grep " r.x. "' > /tmp/maps.txt

The file looks like this:

00010000-00011000 r-xp 00000000 103:04 2617      /usr/bin/WPEWebProcess
...
b54f2000-b6e1e000 r-xp 00000000 103:04 1963      /usr/lib/libWPEWebKit-0.1.so.2.2.1
b6f6b000-b6f82000 r-xp 00000000 00:02 816        /lib/ld-2.24.so
be957000-be978000 rwxp 00000000 00:00 0          [stack]
be979000-be97a000 r-xp 00000000 00:00 0          [sigpage]
be97b000-be97c000 r-xp 00000000 00:00 0          [vdso]
ffff0000-ffff1000 r-xp 00000000 00:00 0          [vectors]

Now we process the backtrace to remove address markers and have one word per line:

$ cat /tmp/bt.txt | sed -e 's/^[^:]*://' -e 's/[<][^>]*[>]//g' | while read A B C D; do echo $A; echo $B; echo $C; echo $D; done | sed 's/^0x//' | while read P; do printf '%08x\n' "$((16#"$P"))"; done | sponge /tmp/bt.txt

Then merge and sort both files, so the addresses in the stack appear below their corresponding mappings:

$ cat /tmp/maps.txt /tmp/bt.txt | sort > /tmp/merged.txt

Now we process the resulting file to get each address in the stack with its corresponding mapping:

$ cat /tmp/merged.txt | while read LINE; do if [[ $LINE =~ - ]]; then MAPPING="$LINE"; else echo $LINE '-->' $MAPPING; fi; done | grep '/' | sed -E -e 's/([0-9a-f][0-9a-f]*)-([0-9a-f][0-9a-f]*)/\1 - \2/' > /tmp/mapped.txt

Like this (address in the stack, page start (or base), page end, page permissions, executable file load offset (base offset), etc.):

0001034c --> 00010000 - 00011000 r-xp 00000000 103:04 2617 /usr/bin/WPEWebProcess
...
b550bfa4 --> b54f2000 - b6e1e000 r-xp 00000000 103:04 1963 /usr/lib/libWPEWebKit-0.1.so.2.2.1
b5937445 --> b54f2000 - b6e1e000 r-xp 00000000 103:04 1963 /usr/lib/libWPEWebKit-0.1.so.2.2.1
b5fb0319 --> b54f2000 - b6e1e000 r-xp 00000000 103:04 1963 /usr/lib/libWPEWebKit-0.1.so.2.2.1
...

The addr2line tool can give us the exact function an address belongs to, or even the function and source code line if the code has been built with symbols. But the addresses addr2line understands are internal offsets, not absolute memory addresses. We can convert the addresses in the stack to offsets with this expression:

offset = address - page start + base offset

I'm using buildroot as my cross-build environment, so I need to pick the library files from the staging directory because those are the unstripped versions. The addr2line tool is the one from the buldroot cross compiling toolchain. Written as a script:

$ cat /tmp/mapped.txt | while read ADDR _ BASE _ END _ BASEOFFSET _ _ FILE; do OFFSET=$(printf "%08x\n" $((0x$ADDR - 0x$BASE + 0x$BASEOFFSET))); FILE=~/buildroot/output/staging/$FILE; if [[ -f $FILE ]]; then LINE=$(~/buildroot/output/host/usr/bin/arm-buildroot-linux-gnueabihf-addr2line -p -f -C -e $FILE $OFFSET); echo "$ADDR $LINE"; fi; done > /tmp/addr2line.txt

Finally, we filter out the useless [??] entries:

$ cat /tmp/bt.txt | while read DATA; do cat /tmp/addr2line.txt | grep "$DATA"; done | grep -v '[?][?]' > /tmp/fullbt.txt

What remains is something very similar to what the real backtrace should have been if everything had originally worked as it should in gdb:

b31ae5ed gst_pad_send_event_unchecked en /home/enrique/buildroot/output5/build/gstreamer1-1.10.4/gst/gstpad.c:5571
b31a46c1 gst_debug_log en /home/enrique/buildroot/output5/build/gstreamer1-1.10.4/gst/gstinfo.c:444
b31b7ead gst_pad_send_event en /home/enrique/buildroot/output5/build/gstreamer1-1.10.4/gst/gstpad.c:5775
b666250d WebCore::AppendPipeline::injectProtectionEventIfPending() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.cpp:1360
b657b411 WTF::GRefPtr<_GstEvent>::~GRefPtr() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/DerivedSources/ForwardingHeaders/wtf/glib/GRefPtr.h:76
b5fb0319 WebCore::HTMLMediaElement::pendingActionTimerFired() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WebCore/html/HTMLMediaElement.cpp:1179
b61a524d WebCore::ThreadTimers::sharedTimerFiredInternal() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WebCore/platform/ThreadTimers.cpp:120
b61a5291 WTF::Function<void ()>::CallableWrapper<WebCore::ThreadTimers::setSharedTimer(WebCore::SharedTimer*)::{lambda()#1}>::call() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/DerivedSources/ForwardingHeaders/wtf/Function.h:101
b6c809a3 operator() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:171
b6c80991 WTF::RunLoop::TimerBase::TimerBase(WTF::RunLoop&)::{lambda(void*)#1}::_FUN(void*) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:164
b6c80991 WTF::RunLoop::TimerBase::TimerBase(WTF::RunLoop&)::{lambda(void*)#1}::_FUN(void*) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:164
b2ad4223 g_main_context_dispatch en :?
b6c80601 WTF::{lambda(_GSource*, int (*)(void*), void*)#1}::_FUN(_GSource*, int (*)(void*), void*) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:40
b6c80991 WTF::RunLoop::TimerBase::TimerBase(WTF::RunLoop&)::{lambda(void*)#1}::_FUN(void*) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:164
b6c80991 WTF::RunLoop::TimerBase::TimerBase(WTF::RunLoop&)::{lambda(void*)#1}::_FUN(void*) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:164
b2adfc49 g_poll en :?
b2ad44b7 g_main_context_iterate.isra.29 en :?
b2ad477d g_main_loop_run en :?
b6c80de3 WTF::RunLoop::run() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:97
b6c654ed WTF::RunLoop::dispatch(WTF::Function<void ()>&&) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/RunLoop.cpp:128
b5937445 int WebKit::ChildProcessMain<WebKit::WebProcess, WebKit::WebProcessMain>(int, char**) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WebKit/Shared/unix/ChildProcessMain.h:64
b27b2978 __bss_start en :?

I hope you find this trick useful and the scripts handy in case you ever to resort to examining the raw stack to get a meaningful backtrace.

Happy debugging!

0 Add to favourites0 Bury

16 Oct 2020 6:07pm GMT

02 Aug 2020

feedPlanet Maemo

Mi Band 5 Review / Mi Band Evolution

Xiaomi has recently released the new Mi Band 5. Since I have owned the each band starting with the Mi Band 2, I think it is time to look back and see where the Mi Band has gone in the recent years.

Actually, the Mi Band story started ahead of the Apple Watch in 2014 with the Mi Band 1, which was a pure fitness-tracking device without a display and even without a heart-beat sensor. This made the device not very appealing to me - even thought it already offered sleep monitoring.

It also already had that interchangeable wrist-bands that allow you to customize the look to your liking. The Mi Band 2 you see in the images uses a custom steel wrist-band as the original one broke after some years of usage.

Below you see a comparison of the Mi Bands, regarding the features that are most significant from my perspective

Mi Band 2

  • Released 2016
  • Clock
  • Heartbeat
  • Notifications

Mi Band 3

  • Released 2018
  • Clock
  • Heartbeat
  • Notifications
  • Timer
  • Weather
  • Workouts

Mi Band 4

  • Released 2019
  • Clock
  • Heartbeat
  • Notifications
  • Timer
  • Weather
  • Workouts
  • Music control

Mi Band 5

  • Released 2020
  • Clock
  • Heartbeat
  • Notifications
  • Timer
  • Weather
  • Workouts
  • Music control
  • Cam shutter

The first thing to note is probably that Xiaomi accelerated the release cycle from 2 years between the Bands 1, 2 and 3 to 1 year between Band 4 and 5. We will come back to this when talking about the Mi Band 5.

Screen legibility comparison

Lets start the comparison with the screen, which is the most obvious part and the one you will probably interact with the most.

Here, the most significant property is neither size nor resolution, but rather legibility in sunlight. For comparison, I set up a little benchmark as follows:

You can find the results below. Also see the banner image for how the screens look indoors.

First I should note that the camera does not do justice to the Mi Band 2 & 3 as their displays are scanline-based and the fast shutter can not capture the whole screen being lit at once. Therefore you only see the top part of the Mi Band 2 and the right part of the Mi Band 3 on the overcast picture.

Nevertheless, one actually cannot read the Mi Band 2 in direct sunlight and only can barely read it in the shade. The other Bands are well readable in the shade. However, I would say that only the MiBand 5 is well readable in direct sunlight.

Next, we will look at how the information is presented. The screen size continuously increased from 0.78″ on the Mi Band 3 to 0.95″ on the Mi Band 4 (+22%) to 1.1″ on the Mi Band 5 (+16%).
As you can read the time on all of them, we will look at an app to find out whether it makes any difference in practice. Here, I picked the weather app as it is probably useful to the majority of the readers.

Looking at the Mi Band 4, it did not really take advantage of the larger screen-estate and shows virtually the same information as the Mi Band 3 - only adding the location info.
The Mi Band 5 on the other hand uses the extra space to show the rain probability. It generally displays more info like the wind strength and the current UV level - however you have scroll down for them.
The Mi Band 2 does not support weather and is thus turned off.

Apps/ on band Screens

Lets also briefly look at the other apps. The images were captured on the Mi Band 5 - however unless otherwise stated the look exactly the same on the Mi Band 4.

Charging

The Mi Band 5 is the first band, with a magnetically attachable charger - hence you do not have to take the band out for charging. This convenience comes at the price of a reduced battery-life from about 20 days with the Mi Band 4 to only 14 days with the Mi Band 5.

As for compatibility, you can charge the Mi Band 2 with the Mi Band 3 charger - the other way round is not possible as the Mi Band 3 is too large for the older charger.

Even though, the Mi Band 4 & 5 have their charging pins at the same location, the chargers are not compatible as the Mi Band 4 lacks the magnetic hold and the Mi Band 5 is too large for the old charger.

The Mi-Fit app

For the Mi Band the accompanying app is quite important as it is the only way to view your sleep data and to monitor your weekly/ monthly stats.

First, lets take a look how you can customize the different Bands from the app. Here, we should note that all bands are still supported by the app.

With the Mi Band, there is only a predefined set of screens/ apps out of which you can pick the ones you want. This is probably the largest difference to a real smart-watch, where you can install additional apps from a store.

With the Mi Band 2, the whole set fits on half a screen and you can only enable/ disable the items.
With the other Bands you can additionally re-order the items, which is quite useful as it allows to choose which item appears first when you swipe up or down on the home screen.

On the Mi Band 5, you can additionally configure which app appears when you swipe left/ and right. This is hard-coded to Music Control (and Ali Pay on the CN version) with the Mi Band 4.

So the basic things work. Lets look at some peculiarities of Mi Fit next.

First you see the workout view for outdoor running, which displays some useful stats like your pace per km and the continuously measured heart-beat rate over time.

What you do not immediately see is that the app only counted ~7.3 km, while my running distance is actually 10 km, which I have verified on google-maps.
One might now think that this is due to imprecise measuring of the band - however on the activity overview, where the daily steps are counted, the running activity is correctly accounted as 10.1 km - which is impressively accurate, given that it only counted the steps.

So the error is only present in the workout app, which is still quite annoying as it also provides the live view during a run.

If someone from Xiaomi is reading this: the error factor of ~0.73 is suspiciously close to the km to miles conversion factor of 0.625.
The error is present with both the Mi Band 4 and Mi Band 5, so I guess it is actually in the App, where I already reported it several times.
If you want happy customers, you better fix this. Many other reviews actually blame this on the band!

Addendum: with the firmware update to v1.0.1.32, the band now measures ~9km which reduces the error factor to 0.9. We are getting there.

So having talked about the bad, lets continue with the ugly. The second screenshot shows you an in-app ad for some obscure Xiaomi product on the home-screen.
These do not show up too often and currently only advertise their own products. However, this is definitely the wrong path you are on.

Ultimately, this leaves me with mixed feelings about Mi Fit. In the Mi Band 2 days it started as a slim and functional app. However, at some point they decided to re-write it with the cards-look and animations. This rewrite moved core views one level down in the menu hierarchy and the added animations actually make the app feel sluggish.

Now, with each Band generation new features appear and are integrated in some sub-menu of the app.
For instance, you get weather-alerts nowadays. However, they are not controlled in the general Band notification settings, but rather in the weather menu.
Therefore, I doubt I would discover them as easily if I would not have watched the app grow.

The good news is that due to the popularity of the Mi Band, there are several alternative apps to try, which I probably will do next.

Mi Band history

In the following, I give a quick outline of how the Mi Band evolved. If you only came here for the Mi Band 5 review, skip forward to the Mi Band 4 section.

The Mi Band 2 was released 2016, about a year after the first Apple Watch launched, which brought the wearable category to the mainstream.
At a price of less then 20€ the Mi Band offered most interesting wearable features to me, like heart-beat measurement, sleep monitoring, forwarding of smartphone notifications and ultimately, simply being a wristwatch.

Also it was an ideal way to try this new wearable thing without spending 350€, that Apple called out.

To my surprise the step-based distance estimation was already accurate back then - except for the actual workout mode, that is - as explained in the Mi Fit section.

Mi Band 3

The larger and brighter screen is the obvious advance of the Mi Band 3. However, the significant part is that it also became a touch-screen - whereas the Mi Band 2 only had the single touch-button. This allowed you swiping forth and back of the screens instead of just cycling through them and it also made virtual buttons possible. These are necessary for starting the stopwatch and timer, which are probably the most important additions for me with the Mi Band 3.

You could also start a selection workouts directly from the watch, instead of going though the app. However, this only included a treadmill mode, while I am interested in outdoor running - so I continued using the activity view for that.

More importantly, it added the weather app. If find this to be surprisingly useful. As with the time - even though you find the same info on your phone - having it at hand is better.

Mi Band 4

Again, the colored screen is the most obvious advance. It does not improve usability in any way though. It displays the same data as the monochrome screen of the Mi Band 3, which is probably more power-efficient. It adds a lot of bling though and is brighter and thus better legible in sunlight.

Speaking of bling, you can install third-party watch-faces now and there is a heap of faces to chose from. Take a look here to get an impression.

Turning to something useful, the touch sensor was noticeably improved. With the Mi Band 3 your swipes were sometimes confused with taps, which does not happen with the Mi Band 4 anymore.

The workout app, now finally included outdoor running, which is still broken though (see Mi App section). This makes the music control app the most important addition for me. At least on android, it works with any music player and allows skipping forward/ back and adjusting volume.
This is quite useful when you play music from your phone at a party or for controlling your Bluetooth headphones.

One can use the same wrist-bands as for the Mi Band 3. This made upgrading for me back then a no-brainer, but is also a strong reason to choose the Mi Band v4 over v5, today.

Mi Band 5

This time, there are no obvious advances and the update is rather evolutionary. It does not mean it is insignificant though as it improves the usability on many levels. If you are new to the Mi Bands, you should pick this one.

The most important one is probably the new magnetic charger. Previously you had to take the "watch" out of the wrist-band to charge, whereas you can simply attach the magnetic charger now.

Next, the screen is slightly brighter which makes a difference in direct sunlight though (see screen comparison section) and also boasts more information.

Finally, the software was also noticeably improved. The band displays generally became more configurable. E.g. the custom left/ right swipes which now give you 4 quick access screens instead of 2. Then, the built-in watch-faces now allow customizing the additional info they display. And it continues with the small things like the configurable alerts in the workouts (although the workout app itself still needs to be fixed).
Also, the selection of predefined watch-faces is vastly better then with the Mi Band 4. On the latter you have a hard time finding a watch-face that is simple and does not feature some animated comic figure screaming at you.
These changes could be provided as an update to the Mi Band 4 as well, but are - at the time of writing - exclusive to the Mi Band 5.

Disclaimer

The Mi Band 5 was provided to me free of charge by banggood.com. So if you liked this review and want to support me consider buying using the following affiliate links:

Mi Band 5
Mi Band 4
Mi Band 3

0 Add to favourites0 Bury

02 Aug 2020 1:37pm GMT

19 Jul 2020

feedPlanet Maemo

Meepo Mini 2 vs. Archos SK8

Having never skateboarded before, I saw the Archos SK8 electric skateboard for about 80€ at a sale and thought why not give it a try. This got me into this whole electric skateboarding thing.

Now that I have some more time at home during the summer, I upgraded to the Meepo Mini 2 and after having driven with it more than 100km, I thought I write down my experiences with the two boards and why I should have gotten the Meepo board from the start.

The competitors

The Meepo Mini 2 and the Archos SK8 are not really competing here, which should be clear looking at their price difference. But for completeness, also take a look at the specs of these two boards:

Meepo Mini 2 Archos SK8
Max. speed 46 km/h 15 km/h
Max. range 18 km 7 km
Max. Weight 136 kg 80 kg
Motor 2 x 540 W 1 x 150 W
Battery 144 Wh 50.4 Wh
Weight 7.4 kg 3.9 kg

Specs comparison

Actually, you can swap the Archos SK8 by any of the unbranded "cheap" Chinese boards that share the same design as the ones sold by Oppikle and Hiriyt.

Here, you might wonder how many Watts you actually need. For this I direct you to the Wikpedia article on bicycle performance that contains some sample calculations (and the formulae) which should roughly hold for electric skateboards as well.

Similarities

Before we dig into the differences, lets first note the similarities aka. the choices I made when picking these specific boards in the first place:

First, both boards are hub-motor driven. I made this choice on purpose, as electric skateboards are not road-legal where I live and hub motors are barely noticeable to the non-practiced eye. This reduces my risk of getting fined for riding one.
However, I would probably generally recommend hub-motors over belt-driven motors nowadays as they require less maintenance (no moving parts), while offering a larger range and allowing pushing the board (belt driven block due to the gear ratio). The latter is especially nice, when you have run out of battery or if you do not want to draw any attention.
When electric skateboards were first introduced by boosted, hub-motors were vastly inferior power-wise but that has changed now.

Next, both boards are of so-called "cruiser-style". This is a size in between a regular skateboard and a long-board. They share a stiff deck and a kicktail with the former, while the use the wheels of the latter.
At this point I should note that I mainly use the boards for leisure instead of a daily commute. This means that I value versatility of the board over comfort of ride.
Here, having a kicktail is a must and rules out long-boards. It allows doing sharp turn, "wheelies" and you are more agile with the short board.
However, you do notice the quality of the pavement very clearly in your feet and being out of the skating age my ankle did hurt the first couple of rides before it got used to it.
So if you want to commute large distances, you should probably get a long-board with a flexi-deck that can cushion away most of the bumps.

Differences

Both boards are of similar length, however the Meepo Mini 2 is considerable wider and heavier. It also has a larger wheel-base.

This results in a better grip and you also feel much more stable on the board. Flipping the board around, you see that the SK8 only uses a single-hub motor while the Mini 2 has two and each of them offers more than 3x the power.

If you do not expect the power or if you enable the pro-mode without being one, the Mini 2 can easily throw you off the board when accelerating or breaking. You can tame it though by using the beginner riding mode if you need to learn how to skate first. You can set the modes for acceleration and breaking separately and I would recommend always using at least the pro mode for breaking and learning to deal with it. In case of an emergency you want to be able to stop in time.

Turning to the SK8, the acceleration is.. meh and so are the breaks - in both of the two riding-modes. The difference between them is merely that the top speed is capped at 10km/h in the low-mode.
But I must say that if you are a beginner this is sufficent; if you do not know how to ride being able to get going and to break are your two primary concerns and the SK8 does deliver here. The main drawback of the Archos SK8 is its tiny battery.

Aside: Li-Ion batteries

At this point we should probably briefly discuss Li-Ion battery technology. Mainly, the following two properties:

So where does this leave us with the SK8? I did about 7 rides, fully-discharging the board (you do not want to stop after 10min, right?). And now the second riding-mode is essentially gone: when I try to accelerate the motors draw so much current, that the voltage drops below a critical level and the board turns off. Depending on what state the controller was in, I have to pair the remote again afterwards.
But it also shows in the first mode: while the board initially could get me up a slight slope, it now immediately starts beeping due to critical voltage - again the motors need more voltage then the already worn down battery can give.

Remotes & charging indicators

Having covered the drive train, lets turn to the remotes. Both Archos and Meepo use a similar pistol-grip like design, where you control the motors with the thumb switch.

As one would expect, the Meepo remote is more sophisticated and offers detailed telemetry data on a nicely readable LCD display. There you find your current speed, drive mode and board charging level as well as the max. speed of the current ride.

On the Archos remote you only find 4 LEDs. Those are used quite well though: when you turn on the remote, they show the remote charging level. As soon as the board is connected, they indicate the board charging level, which is actually the most important information you need while riding.

Similarly to the remotes, there are only 4 LEDs on the Archos Battery for the charging level, while you find a numeric LED-display on the Meepo board.

An actually noticeable feature on the Meepo Mini 2 is push-to-start; that is, you only have to push the board to turn it on - no need to bend down for flipping a switch.

Verdict

So why do I say you should go straight with the Meepo Mini 2 even as a beginner? On paper the Archos SK8 has everything it takes to be a nice beginner board.
It is really the battery that kills it. With only 5-10 rides it is simply not worth the money, no matter how cheap it is.
Looking at the price difference between the Meepo Mini 2 and the ER version that solely differ in the battery, you grasp that the battery is the crucial part in an electric skateboard. And the Archos SK8 is cheap, precisely because of the bad battery.

With the Meepo Mini 2 on the other hand you get a board that can "grow with you": as you get more confident you can bump up the riding mode to get more power. Even if you decide that skateboarding is not for you, you can sell the Mini 2 as it will retain lots of its value - in contrast to just producing electric waste with the Archos SK8.

Riding the Meepo Mini 2

The Meepo Mini 2 is specified to go up to 46 km/h. Whether you can go that fast depends on your weight, the wind and the slope (see the Wikipedia link, mentioned above). In case you are fat and/ or there are lots of slopes where you live, you might also consider the ER version of the Mini 2, which comes with doubled battery capacity. As mentioned above this not only means that you can get further, but also that you have more power in the mid-range.

How fast can you go?

Having only previous experience on a Snowboard, I am a rather cautious rider. So far my max. speed (according to the remote) was 30 km/ h which I did uphill - in hope that stopping is easier that way.
Going downhill (only using the motors for breaking), I feel comfortable until around 22 km/ h.
I typically ride for about 30-45 min and the lowest the battery got was 40%, which means it should last for quite some time.
Note, that I do not go straight uphill for 30min and that I usually push to get rolling, as this is where most energy is used.

A suitable helmet

When lifting the board, the remote showed that the ESC only limits the speed at 50 km/h. When riding a skateboard at anything above 10 km/h without a cushion-zone and no nothing, I would highly recommend you to at least wear a helmet.
However, you should consider that a "normal" skate helmet is only specified (EN1078) up to 19.5 km/h impact speed - if you ride faster it does not guarantee protection.

Fortunately, one does not have to resort to heavy motorcycle helmets (ECE2205) as there is a specification (ASTM F1952) for downhill bike helmets, which was designed with speed in mind. It is specified for an impact speed of 22.3 km/h and typically the helmets provide full-face protection.
Recently, even variants for downhill longboarding started to pop-up, which should be even a better fit style-wise.

Addendum: replaced NTA8776 by ASTM F1952 in the last paragraph

0 Add to favourites0 Bury

19 Jul 2020 8:58pm GMT

30 Jun 2020

feedPlanet Maemo

Developing on WebKitGTK with Qt Creator 4.12.2

After the latest migration of WebKitGTK test bots to use the new SDK based on Flatpak, the old development environment based on jhbuild became deprecated. It can still be used with export WEBKIT_JHBUILD=1, though, but support for this way of working will gradually fade out.

I used to work on a chroot because I love the advantages of having an isolated and self-contained environment, but an issue in the way bubblewrap manages mountpoints basically made it impossible to use the new SDK from a chroot. It was time for me to update my development environment to the new ages and have it working in my main Kubuntu 18.04 distro.

My mail goal was to have a comfortable IDE that follows standard GUI conventions (that is, no emacs nor vim) and has code indexing features that (more or less) work with the WebKit codebase. Qt Creator was providing all that to me in the old chroot environment thanks to some configuration tricks by Alicia, so it should be good for the new one.

I preferred to use the Qt Creator 4.12.2 offline installer for Linux, so I can download exactly the same version in the future in case I need it, but other platforms and versions are also available.

The WebKit source code can be downloaded as always using git:

git clone git.webkit.org/WebKit.git

It's useful to add WebKit/Tools/Scripts and WebKit/Tools/gtk to your PATH, as well as any other custom tools you may have. You can customize your $HOME/.bashrc for that, but I prefer to have an env.sh environment script to be sourced from the current shell when I want to enter into my development environment (by running webkit). If you're going to use it too, remember to adjust to your needs the paths used there.

Even if you have a pretty recent distro, it's still interesting to have the latests Flatpak tools. Add Alex Larsson's PPA to your apt sources:

sudo add-apt-repository ppa:alexlarsson/flatpak

In order to ensure that your distro has all the packages that webkit requires and to install the WebKit SDK, you have to run these commands (I omit the full path). Downloading the Flatpak modules will take a while, but at least you won't need to build everything from scratch. You will need to do this again from time to time, every time the WebKit base dependencies change:

install-dependencies
update-webkitgtk-libs

Now just build WebKit and check that MiniBrowser works:

build-webkit --gtk
run-minibrowser --gtk

I have automated the previous steps as go full-rebuild and runtest.sh.

This build process should have generated a WebKit/WebKitBuild/GTK/Release/compile_commands.json
file with the right parameters and paths used to build each compilation unit in the project. This file can be leveraged by Qt Creator to get the right include paths and build flags after some preprocessing to translate the paths that make sense from inside Flatpak to paths that make sense from the perspective of your main distro. I wrote compile_commands.sh to take care of those transformations. It can be run manually or automatically when calling go full-rebuild or go update.

The WebKit way of managing includes is a bit weird. Most of the cpp files include config.h and, only after that, they include the header file related to the cpp file. Those header files depend on defines declared transitively when including config.h, but that file isn't directly included by the header file. This breaks the intuitive rule of "headers should include any other header they depend on" and, among other things, completely confuse code indexers. So, in order to give the Qt Creator code indexer a hand, the compile_commands.sh script pre-includes WebKit.config for every file and includes config.h from it.

With all the needed pieces in place, it's time to import the project into Qt Creator. To do that, click File → Open File or Project, and then select the compile_commands.json file that compile_commands.sh should have generated in the WebKit main directory.

Now make sure that Qt Creator has the right plugins enabled in Help → About Plugins…. Specifically: GenericProjectManager, ClangCodeModel, ClassView, CppEditor, CppTools, ClangTools, TextEditor and LanguageClient (more on that later).

With this setup, after a brief initial indexing time, you will have support for features like Switch header/source (F4), Follow symbol under cursor (F2), shading of disabled if-endif blocks, auto variable type resolving and code outline. There are some oddities of compile_commands.json based projects, though. There are no compilation units in that file for header files, so indexing features for them only work sometimes. For instance, you can switch from a method implementation in the cpp file to its declaration in the header file, but not the opposite. Also, you won't see all the source files under the Projects view, only the compilation units, which are often just a bunch of UnifiedSource-*.cpp files. That's why I prefer to use the File System view.

Additional features like Open Type Hierarchy (Ctrl+Shift+T) and Find References to Symbol Under Cursor (Ctrl+Shift+U) are only available when a Language Client for Language Server Protocol is configured. Fortunately, the new WebKit SDK comes with the ccls C/C++/Objective-C language server included. To configure it, open Tools → Options… → Language Client and add a new item with the following properties:

Some "LanguageClient ccls: Unexpectedly finished. Restarting in 5 seconds." errors will appear in the General Messages panel after configuring the language client and every time you launch Qt Creator. It's just ccls taking its time to index the whole source code. It's "normal", don't worry about it. Things will get stable and start to work after some minutes.

Due to the way the Locator file indexer works in Qt Creator, it can become confused, run out of memory and die if it finds cycles in the project file tree. This is common when using Flatpak and running the MiniBrowser or the tests, since /proc and other large filesystems are accessible from inside WebKit/WebKitBuild. To avoid that, open Tools → Options… → Environment → Locator and set Refresh interval to 0 min.

I also prefer to call my own custom build and run scripts (go and runtest.sh) instead of letting Qt Creator build the project with the default builders and mess everything. To do that, from the Projects mode (Ctrl+5), click on Build & Run → Desktop → Build and edit the build configuration to be like this:

Then, for Build & Run → Desktop → Run, use these options:

With these configuration you can build the project with Ctrl+B and run it with Ctrl+R.

I think I'm not forgetting anything more regarding environment setup. With the instructions in this post you can end up with a pretty complete IDE. Here's a screenshot of it working in its full glory:

Anyway, to be honest, nothing will ever reach the level of code indexing features I got with Eclipse some years ago. I could find usages of a variable/attribute and know where it was being read, written or read-written. Unfortunately, that environment stopped working for me long ago, so Qt Creator has been the best I've managed to get for a while.

Properly configured web based indexers such as the Searchfox instance configured in Igalia can also be useful alternatives to a local setup, although they lack features such as type hierarchy.

I hope you've found this post useful in case you try to setup an environment similar to the one described here. Enjoy!

0 Add to favourites0 Bury

30 Jun 2020 2:47pm GMT

22 Jun 2020

feedPlanet Maemo

PhotoTeleport 0.13

Just a quick note to let the world know that PhotoTeleport 0.13 has been released.

0 Add to favourites0 Bury

22 Jun 2020 8:11pm GMT

15 May 2020

feedPlanet Maemo

Lecture on Augmented Reality

Due to the current circumstances, I had to record the lectures on augmented reality, which I am typically holding live. This was much more work than anticipated..
On the other hand, this means that I can make them available via Youtube now.

So, if you ever wanted to learn about the basic algorithms behind Augmented Reality, now is your chance.

The lecture is structured in two parts

You can click on the TOC links below to directly jump to a specific topic.

Camera Calibration

Object Pose Estimation

0 Add to favourites0 Bury

15 May 2020 2:57pm GMT

24 Apr 2020

feedPlanet Maemo

Error handling and exceptions

Yes, this is yet another post in the internet talking about using exceptions versus error returns. The topic has been flaming up at my workplace for quite some time now, and I felt that writing a blog post about it during the week-end would help me focus my thoughts and give me time to explain my point with the due care. In case you didn't know, I'm against using exceptions for error handling (maybe having spent many years working with Qt has had an effect on this); that does not mean that I never write code using exceptions: I certainly do my good share of try ... catch when dealing with third-party code (including the STL), but you won't find a throw in my programs.

I'm not going to write here all the reasons why I refrain myself from implementing error handling using exceptions; I'd rather like to focus on the one I consider to be the major one, and which I rarely see being given the due weight in the debate.

And please note that this post is about C++ only; it may be that exception handling in other languages is designed in such a way that all my concerns are addressed (either by the language itself, or by common error handling policies).

Code safety

I was about to title this "Code readability", but this is more about code verifiability, that is making sure that the code is correct and, ultimately, safe. As we all know, code is written once but read many times, and even if it's code you've written yourself, chances are that in a few weeks time you'll have forgotten several details about it; error cases and error handling are one typical thing that doesn't stick in our memory for long.

When I look at a small piece of code, such as the one that can fit into my screen, or which I can read from a merge request diff, I want to be able to ascertain that the code I'm looking at is correct. Let's look at some examples.

A throw-free project

assert(track != nullptr);

Car car;
car.setMaximumSpeed(90);
car.setName("Herbie");

if (!car.executeLap(track)) {
    log("Car failed to complete track");
    return false;
}

Path *path = car.getPath();
if (!path) {
    log("GPX path could not be retrieved");
    return false;
}

double temperature = car.engineTemperature();
double boundingRectArea = path->boundingRectArea();

I just made this up, so please bear with me if it doesn't make any sense. What I want to show is that code like the above has very few fault risks, if found in a project which bans throwing errors as exceptions: if we exclude out-of-memory errors, that are generally not handled to let the application crash (though you can always catch them if you like), the reader can easily verify that this code is safe. Coding style policies and naming conventions can guarantee that setMaximumSpeed() and setName() won't have a return value that needs to be checked, and all other method calls either return an error that our code is properly handling, or return some value. Of course, by just looking at this piece of code we cannot know if the engineTemperature() method has some other overloaded sibling which accepts passing a reference to a boolean and which could be used to detect an error; so, it may be that our code could be improved in that respect, if we had a look at the header files for the Car class - but this does deny the fact that a simple glance at this snippet tells us exactly what errors are handled and what could be going wrong.

Let's look at this code instead:

assert(track != nullptr);

Car car;
car.setMaximumSpeed(90);
car.setName("Herbie");

car.executeLap(track);
Path *path = car.getPath();

double temperature = car.engineTemperature();
double boundingRectArea = path->boundingRectArea();

If we continue on the assumption that we are working on a project which bans throwing exceptions, we can immediately say that this code is not safe: we don't know if the car successfully executed a lap on the track, and our process will crash if boundingRectArea() is invoked on a null object.

Enter the exception

In a project where exceptions are actively used, the code from the second snippet is not obviously wrong anymore: maybe executeLap() cannot throw any exceptions, or, if does, the caller of this snippet is catching the exception? In order to figure out whether this code is correct, I need to see the declaration of the executeLap() method, and hope that there's a nice noexcept in there; if there isn't, I have to look at its implementation, and recursively descend through all the methods it calls - at which point the safest attitude is just to assume that it can throw. But that's only half of the story, because once I accept the fact that executeLap() can throw, I need to check whether the exception is properly handled: I have to check the implementation of all the callers of my method, and if I don't find a catch there, I'll have to recursively walk up the tree of their callers.

And indeed even the first snippet, which looked so harmless when exception throwing was banned, suddenly becomes not obviously correct anymore: what if executeLap() or getPath() also throw an exception? You might say that it would be quite a silly thing to do, and I'd certainly agree; but it may be that indeed they don't throw any exceptions in their implementation, but some of the methods they call does.

A compromise: catch early, catch often

The obvious solution to the above issue is having a policy of handling exceptions right away, and explicitly rethrowing them (or even better, rethrow a different, more appropriate exception) up the stack:

assert(track != nullptr);

Car car;
car.setMaximumSpeed(90);
car.setName("Herbie");

try {
    car.executeLap(track);
    Path *path = car.getPath();

    double temperature = car.engineTemperature();
    double boundingRectArea = path->boundingRectArea();
} catch (std::runtime_error &e) {
    log("Car failed to complete track");
    throw;
}

What I can tell from the above snippet is that the code is handling errors, and this is somehow a relief. I'm sure some of you would suggest using a more specific catch clause, but for the sake of this example let's assume that this one is fine.

(Quick note: the above example does not catch std::exception, because that would also catch the std::bad_alloc exception which is typically thrown in out-of-memory situations; my advice is not handle it at all, unless you know what you are doing)

In real life, though, you might find that try-ing on a rather large block of operations is not enough: suppose that the Car methods all emit the same exception type, and that you need to handle them differently depending on when they occur. Then you'd need to split up the try into smaller blocks, and at that point your code won't look any cleaner than the equivalent code which uses ifs on return values. Of course if you own the Car class you could modify it to throw different exceptions, in order to keep more operations inside the try block and have specific catches at the end.

The big catch (pun intended)

Even once you've refactored your methods to get the best out of exceptions (where "best" is highly subjective, but let's assume that it just means that you are happy with your exception-throwing code), there's something that still bothers me, and that's exactly the same thing that proponents of exceptions use as a "pro" in their argumentations: the business logic of your code gets separated from the error handling. You get a nice block of pure logic, not cluttered with error checking, and a catch section (which I call "the big catch") where error cases are handled.

I really don't see how that makes the code any more readable or safe: sure, the logic is not intertwined with error handling and might help focus on the expected flow of the operations (though, really, I do not think that normal brains have a problem skipping over if blocks), but that's hardly what I'm interested in when I want to check that the code is correct. Most of program errors and bugs lie in handling the edge cases and the abnormal situations, the seldomly taken code paths, and that's where I need to focus my attention.

try {
    operationA();
    if (value > B.maxValue()) {
        operationB();
    } else {
        operationC();
    }
    operationD();
} catch (ExceptionI &e) {
    ...
} catch (ExceptionII &e) {
    ...
} catch (ExceptionIII &e) {
    ...
} catch (std::runtime_error &e) {
    ...
}

When I see code like this one, I need to mentally build a mapping of "operationX() → possible exceptions" (which, unless exception naming is making this obvious, requires me to look at the implementation of the operationX() functions), and then mentally reconstruct the possible code paths in case operationX() fails, for each line of the try block.

Not seeing the errors right there, right away makes the correctness verification harder, which in turns means that the code becomes less safe. It will make you focus on the best case scenario, while ignoring all those annoying edge cases - too bad that 90% of the bugs are there.

Reading through the ISO C++ propaganda FAQ

I've been given a link to the C++ FAQ about exceptions, and unfortunately I read it. While there isn't much to argue on the technical side of it, it also carries some misleading statements, which might be true in absolute terms but don't let you see the big picture by not mentioning all that you need to know (which is the fundamental technique behind propaganda). An example is when they mention that eliminating ifs makes for more robust code, without mentioning that the same applies to all code branches, including exceptions.

Another argument that bothered me when I read it is the one about error propagation; this is the example they make:

void f1()
{
    try {
        // ...
        f2();
        // ...
    } catch (some_exception& e) {
        // ...code that handles the error...
    }
}
void f2() { ...; f3(); ...; }
void f3() { ...; f4(); ...; }
void f4() { ...; f5(); ...; }
void f5() { ...; f6(); ...; }
void f6() { ...; f7(); ...; }
void f7() { ...; f8(); ...; }
void f8() { ...; f9(); ...; }
void f9() { ...; f10(); ...; }
void f10()
{
    // ...
    if ( /*...some error condition...*/ )
        throw some_exception();
    // ...
}

The claim is that this code is more readable than the one with explicit error handling, because all the f2(), f3, …, f9() functions don't have to handle the error occurring in f10(). It is indeed a convincing argument, when presented in these terms, but is this really how our code looks like? In real life, you'll hardly have a chain of 1-liner functions, all defined next to each other in the same file. The moment that you realize that each one of these fn() functions might be twenty or thirty lines long, and that they might be scattered over different files, and be called not just by fn-1() but by any other function in the codebase, the picture does not look so rosy anymore: we get back to my main point of pain, that is that looking at the code of, say, f5(), I will not be able to tell if the errors thrown by it, or by any of the methods invoked by it, are properly handled.

Exceptions in APIs

A side note about projects using exceptions. I'm not really bothered when a library I need to use is throwing exceptions: having to write

try {
    Foo::fetch("http://example.com/resource.txt");
} catch (Foo::Exception &) {
    return false;
}

is not less readable or less safe than the code I'd write if Foo::fetch() returned an error code. I still do have a little complaint, because the library author has given himself the right to decide that a failure in his library should be considered a critical fault, whereas it may be that in my program it is an expected failure and using exceptions imposes a penalty which could have been avoided. But I digress.

As long as the library documents which exceptions are thrown, it is used by many people (which hopefully means that it has few bugs) and it is a library that I don't need to contribute to, wrapping some of its methods in try blocks is something I can live with.

One situation where I actually wish that libraries threw an exception is in out-of-memory situations; in that case, of course, I'd expect them to throw nothing else than std::bad_alloc, which is the exception emitted by the standard library in such situations. That allows the caller to decide whether to ignore the exception and have the process terminated (which is what I usually do, at least in desktop applications) or try their luck and handle the failure - the latter is not easy, but it can certainly be done.

This is one case where error returns can be problematic, because it's likely that your code would look something like

if (!Foo::open(fileName)) {   // suppose that this returns Error::OutOfMemory
    log("Failed to open " << fileName);
    return false;
}

and in this case there's actually a risk that your code is going to trigger an out-of-memory error in logging the message; this shouldn't be a concern in most cases, but I can imagine some situations where one might want to know which was the exact operation that first incurred in the out-of-memory failure.

So, I'm actually fine with new throwing. As for my code, my throw statement is actually spelt as return.

0 Add to favourites0 Bury

24 Apr 2020 3:07pm GMT

12 Apr 2020

feedPlanet Maemo

New website for Mappero Geotagger, and cross-compiling stuff

Mappero Geotagger has now moved from its previous page from this site to a new, separate website built with the awesome Nikola static website generator.

The main reason for this change is that I didn't have an online space where to host the application binaries, and I wanted to experiment with a different selling method. Now, downloads are (poorly) hidden behind a payment page, whereas in multiple places of the website I also mention that I can provide the application for free to whomever asks for it. While it might seem weird at first, I do honestly believe that this will not stop people from buying it: first of all, many people just think it's fair to pay for a software applications, and secondly, for some people writing an e-mail and establishing a personal contact with a stranger is actually harder than paying a small amount of money. And in all sincerity, the majority of the income I've had so far for Mappero Geotagger came from donations, rather than purchases; so, not much to lose here.

QBS and MXE

Anyway, since this is primarily a technical blog, I want to share my experiences with cross-building from Linux to Windows. As you might remember, some time ago I switched the build system of Mappero from qmake to QBS, and I haven't regretted it at all. I've managed to build the application in Linux (of course), macOS, as a Debian package on the Ubuntu PPA builders, on Windows with AppVeyor and, last but not least, on Linux for Windows using the mingw setup provided by the MXE project.

QBS worked surprisingly well also in this case, though I had to fight with a small bug on the toolchain detection, which is hopefully going to be fixed soon. For the few of you who are interested in achieving something similar, here's the steps I ran to configure QBS for mingw:

    MXE_BASE=<path-to-mxe>
    MXE_TARGET=x86_64-w64-mingw32.shared # 32 bit or static targets are also available

    MXE_PROFILE="mxe"
    QT_PROFILE="${MXE_PROFILE}-qt"
    qbs setup-toolchains "${MXE_BASE}/usr/bin/${MXE_TARGET}-g++" $MXE_PROFILE
    qbs config profiles.$MXE_PROFILE.cpp.toolchainPrefix "${MXE_TARGET}-" # temporary workaround
    qbs setup-qt "$MXE_BASE/usr/$MXE_TARGET/qt5/bin/qmake" ${QT_PROFILE}
    qbs config profiles.${QT_PROFILE}.baseProfile $MXE_PROFILE

Sorry for using that many environment variables ☺. After qbs is configured, it's just a matter of running

    qbs profile:$QT_PROFILE

to build the application. You will get a nice window binary and, once you collect all the needed library dependencies, you'll be able to run it on Windows. Or WINE ☺.

As part of this effort, I also had to build libraw, so I didn't miss the occasion to contribute its recipe to MXE. I'm also trying to get a change accepted, that would make MXE support the dynamic OpenGL selection available since Qt 5.4.

0 Add to favourites0 Bury

12 Apr 2020 9:20am GMT

23 Mar 2020

feedPlanet Maemo

Are we dead yet?

I am quite frustrated with corona graphs in the news, since most reporters seem to have skipped math classes back then. For instance, just plotting the number of confirmed infections at the respective dates does not tell you anything due to the different time point of outbreak. So lets see whether I can do better:

https://paroj.github.io/arewedeadyet/

With the site above, I tried to improve on a few things:

0 Add to favourites0 Bury

23 Mar 2020 11:40am GMT