04 Jul 2019
Planet Maemo
Notes on porting the Samsung J3 to Halium + Ubports
A due premise: this is not a guide! While at the end I managed to build an image for my phone, it doesn't even boot. So, this post should be taken just as a collection of notes for myself, written to be able to reproduce the build and keep track of those steps I might want to revisit later.
The reason why I'm publishing these notes in my blog is that I believe they can still be useful for other people who should stumble into the same errors, and also to give you an idea of how much fun (or lack thereof) porting is.
It's a very long post, but unless you are starting to port Halium or Ubports to a new device, you can safely stop reading now.
I hope to write some better porting news in some future post. :-)
Before we start: getting the info
The first step is finding out a few informations about our device, and whether a LineageOS port for our device exists (hadn't there been a LineageOS port, I would have been out of luck: I certainly wouldn't have the time to do one myself). From the Settings application we can find the device name, the model number, the Android version and the kernel version:
- Device name: "Samsung Galaxy J3 (2016)"
- Model: SM-J320F
- Android version: 5.1.1
- Kernel version: 3.10.65-[…]
With this information we can look for a LineageOS port, either via a search engine or directly in the XDA forums; in my case the thread is this one. The post announcing the port should contain links to the source code, besides some other information. We do especially care about:
- the device codename:
j3xlte - the CM/LineageOS version: 14.1
- links to Device Tree, Kernel, Local Manifest (keep the local manifest handy, as you are going to need it soon)
To be on the safe side, get a copy of everything. In case these are GitHub repositories, forking them is enough.
Getting root access on the device
Another precondition for a successful port is the ability to be able to flash a custom kernel and rootfs image. The existance of a LineageOS port is not a guarantee that we can flash it: some devices are sold with a locked bootloader in certain markets or when sold by certain mobile operators. So, before spending time on the port, we should make sure that we can hack on our device freely.
In order to get adb working, I had to enable "Developer mode", which is done by hitting the "Build numer" item under the "Software info" page for 10 times in a row. Then from the newly appeared "Developer options" menu I enabled "USB debugging".
Being a total novice to Android phones, I had to go through several attempts and internet searches, before being able to get root access on the device (which I needed in order to read the /etc/fstab file, for example): by default, adb only let me as normal user, and su didn't work.
In order to flash a rootkit on this phone, we need to use Heimdall, because on Samsung devices this is what is used instead of fastboot. I just followed the instructions in the Linux/README file and could easily build and install it.
We are going to flash TWRP and SuperSU; a link to the former can be found in the XDA post about the LineageOS port for our device (under the "Download" button) and the latter from its own homepage (in my case I had to download the "Recovery flashable" zip file, and for some weird reason I got the 2.76 version, but I bet that any newer version works as well). Once I got these files, I followed these steps:
- Extract TWRP in my host machine:
cd /tmp && tar xvf ~/Downloads/sm-j320fgm_twrp_3.1.1_f2fs.tar.md5(that leaves arecovery.imgfile in my/tmp) - Upload SuperSU to the phone:
adb push ~/Downloads/UPDATE-SuperSU-v2.76-20160630161323.zip /sdcard/supersu.zip
- Power off the phone
- Power it on by keeping the VOLUME DOWN and HOME keys pressed while pressing the power button. The phone will be brought up into Odin mode.
- Run
sudo heimdall flash --RECOVERY /tmp/recovery.img
-
Before the phone reboots, start keeping the VOLUME UP (note: it's not the same as before!) and HOME keys, to make it reboot into recovery and not into the ordinary image. If the phone boots into the ordinary image it will overwrite the TWRP we just flashed, so, if that happens, you need to start from scratch.
-
Find supersu (the zip file) and install it.
If you reboot the phone and use adb shell, you'll be able to run su and get root rights. At this point, I charged myself with hopes and got to the real work.
Let the Odyssey start
I just followed the "First steps" instructions step by step, with no major surprises.
Once done with that, I moved to the "Get sources" page. Here it was not just a matter of following the instructions: assembling the manifest file took me several iterations, so I had to come back to this step a few times. Anyway, I followed the instructions for the halium-7.1 version, given that my device has a LineageOS 14.1 port. Then:
- I created a git branch in the
halium/devices/tree, to keep track of my changes - I added the manifest
halium/devices/manifests/samsung_j3xlte.xml(you can see the final version in my repository, but initially I was missing some important lines) - Run
JOBS=2 ./halium/devices/setup j3xlte - The makefile needed to be changed: the
device/samsung/j3xlte/setup-makefiles.shscript (from the device tree) was missing amkdir -p ../../../$OUTDIRnear the beginning.
At this point I continued to the "Build sources" page. Here starts a long trial and error phase, and the longer part of this post. For the sake of documentation, I will show you all the commands I ran, including failed attempts (there are many of them), just to give you an idea of what problems you might run into with your own port, and hopefully help in finding a solution.
I first ran the breakfast command for my device:
$ breakfast j3xlte including vendor/cm/vendorsetup.sh build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/samsung/sharkls-common/sharkls.mk" does not exist. Stop. build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/samsung/sharkls-common/sharkls.mk" does not exist. Stop. build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/samsung/sharkls-common/sharkls.mk" does not exist. Stop. Device j3xlte not found. Attempting to retrieve device repository from LineageOS Github (http://github.com/LineageOS). Repository for j3xlte not found in the LineageOS Github repository list. If this is in error, you may need to manually add it to your local_manifests/roomservice.xml. build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/samsung/sharkls-common/sharkls.mk" does not exist. Stop. build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/samsung/sharkls-common/sharkls.mk" does not exist. Stop. ** Don't have a product spec for: 'lineage_j3xlte' ** Do you have the right repo manifest?
It looks like the device/samsung/sharkls-common was not checked out properly. After re-doing with force-sync, it failed with other errors:
$ breakfast j3xlte including vendor/cm/vendorsetup.sh build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/common/gps/gps_eu_supl.mk" does not exist. Stop. build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/common/gps/gps_eu_supl.mk" does not exist. Stop. build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/common/gps/gps_eu_supl.mk" does not exist. Stop. Device j3xlte not found. Attempting to retrieve device repository from LineageOS Github (http://github.com/LineageOS). Repository for j3xlte not found in the LineageOS Github repository list. If this is in error, you may need to manually add it to your local_manifests/roomservice.xml. build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/common/gps/gps_eu_supl.mk" does not exist. Stop. build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/common/gps/gps_eu_supl.mk" does not exist. Stop. ** Don't have a product spec for: 'lineage_j3xlte' ** Do you have the right repo manifest?
Indeed, I had nothing under device/common. So, I looked under halium/devices/manifests to see if any other manifest was populating this directory, and found that the bq_krillin.xml file had such a line. So, I copied that line (the one adding the lineageos/android_device_common project) into my manifest. After doing that, the breakfast command succeeded:
$ breakfast j3xlte including vendor/cm/vendorsetup.sh Trying dependencies-only mode on a non-existing device tree? ============================================ PLATFORM_VERSION_CODENAME=REL PLATFORM_VERSION=7.1.1 LINEAGE_VERSION=14.1-20190619-UNOFFICIAL-j3xlte TARGET_PRODUCT=lineage_j3xlte TARGET_BUILD_VARIANT=userdebug TARGET_BUILD_TYPE=release TARGET_BUILD_APPS= TARGET_ARCH=arm TARGET_ARCH_VARIANT=armv7-a-neon TARGET_CPU_VARIANT=cortex-a7 TARGET_2ND_ARCH= TARGET_2ND_ARCH_VARIANT= TARGET_2ND_CPU_VARIANT= HOST_ARCH=x86_64 HOST_2ND_ARCH=x86 HOST_OS=linux HOST_OS_EXTRA=Linux-4.15.0-51-generic-x86_64-with-Ubuntu-16.04-xenial HOST_CROSS_OS=windows HOST_CROSS_ARCH=x86 HOST_CROSS_2ND_ARCH=x86_64 HOST_BUILD_TYPE=release BUILD_ID=NOF27B OUT_DIR=/mnt/Lavoro/halium/out RECOVERY_VARIANT=twrp WITH_SU=true ============================================
Building the kernel
According to the livestream, the proper script to check for the kernel config is not the Mer one, but halium/halium-boot/check-kernel-config (this is also documented in the Ubports - Building Halium boot documentation page).
At this point I realized that I didn't have a kernel: an entry starting with kernel/ should always be in the manifest file. So I added the kernel line taken from the LineageOS local manifest file.
I then looked for the config file, which I eventually found in arch/arm/configs/j3xlte_defconfig. So I ran
./halium/halium-boot/check-kernel-config kernel/samsung/sharkls/arch/arm/configs/j3xlte_defconfig -w
which reported no errors. However, in the command output I noticed this:
CONFIG_TMPFS_POSIX_ACL is already set CONFIG_DEFAULT_SECURITY is set, but to y "selinux" y not "apparmor". Setting CONFIG_DEFAULT_SECURITY="apparmor" correctly sed: -e expression #1, char 60: unterminated `s' command Setting CONFIG_SECURITY_APPARMOR_BOOTPARAM_VALUE=1
When I checked the config file, I noticed that AppArmor was not really enabled. So I changed it manually.
Next, the fstab. I had the Ubports and Halium guides both opened in my browser, and I went for the "Fix mounts" step from the Ubports guide. I used find to locate the fstab, and found two:
device/samsung/sharkls-common/recovery/root/fstab.sc8830device/samsung/sharkls-common/rootdir/etc/fstab.sc8830
The recovery file seemed already fine to me, while the rootdir one had most entry duplicated; ones with f2fs and xattrs, the other ones with ext4. I commented out the f2fs ones and ran mka halium-boot:
[...] /mnt/Lavoro/halium/out/build-aosp_arm.ninja is missing, regenerating... ============================================ PLATFORM_VERSION_CODENAME=REL PLATFORM_VERSION=7.1.1 LINEAGE_VERSION= TARGET_PRODUCT=aosp_arm TARGET_BUILD_VARIANT=eng TARGET_BUILD_TYPE=release TARGET_BUILD_APPS= TARGET_ARCH=arm TARGET_ARCH_VARIANT=armv7-a TARGET_CPU_VARIANT=generic TARGET_2ND_ARCH= TARGET_2ND_ARCH_VARIANT= TARGET_2ND_CPU_VARIANT= HOST_ARCH=x86_64 HOST_2ND_ARCH=x86 HOST_OS=linux HOST_OS_EXTRA=Linux-4.15.0-51-generic-x86_64-with-Ubuntu-16.04-xenial HOST_CROSS_OS=windows HOST_CROSS_ARCH=x86 HOST_CROSS_2ND_ARCH=x86_64 HOST_BUILD_TYPE=release BUILD_ID=NOF27B OUT_DIR=/mnt/Lavoro/halium/out ============================================ Checking build tools versions... find: 'device/*/generic': No such file or directory find: 'device/unknown': No such file or directory find: 'device/android': No such file or directory find: 'device/*/generic': No such file or directory find: 'device/unknown': No such file or directory find: 'device/android': No such file or directory halium/hybris-boot/Android.mk:69: ********************* /boot appears to live on ERROR: *fstab* not found halium/hybris-boot/Android.mk:70: ********************* /data appears to live on ERROR: *fstab* not found halium/hybris-boot/Android.mk:73: *** There should be a one and only one device entry for HYBRIS_BOOT_PART and HYBRIS_DATA_PART. build/core/ninja.mk:166: recipe for target '/mnt/Lavoro/halium/out/build-aosp_arm.ninja' failed make: *** [/mnt/Lavoro/halium/out/build-aosp_arm.ninja] Error 1 make: Leaving directory '/mnt/Lavoro/halium' #### make failed to build some targets (11 seconds) ####
this seems to be documented in the "Common system build errors" page: I had to rerun source build/envsetup.sh and breakfast j3xlte. After that, the mka halium-boot command failed with
device/samsung/sharkls-common/ims/sec_samsung/Android.mk:25: build/core/java_library.mk: No such file or directory build/core/ninja.mk:166: recipe for target '/mnt/Lavoro/halium/out/build-lineage_j3xlte.ninja' failed make: *** [/mnt/Lavoro/halium/out/build-lineage_j3xlte.ninja] Error 1 make: Leaving directory '/mnt/Lavoro/halium'
which looks like a progress. The issue about the missing java_library.mk has been raised in the #halium channel; possible solutions include:
- symlink from
static_java_library.mk - edit the
Android.mkfile
I went for the former; this probably needs to be fixed in Halium.
Reran mka halium-boot, got some improvement: fstab is found:
halium/hybris-boot/Android.mk:70: ********************* /boot appears to live on halium/hybris-boot/Android.mk:71: ********************* /data appears to live on /dev/block/platform/sdio_emmc/by-name/userdata halium/hybris-boot/Android.mk:74: *** There should be a one and only one device entry for HYBRIS_BOOT_PART and HYBRIS_DATA_PART. build/core/ninja.mk:166: recipe for target '/mnt/Lavoro/halium/out/build-lineage_j3xlte.ninja' failed make: *** [/mnt/Lavoro/halium/out/build-lineage_j3xlte.ninja] Error 1 make: Leaving directory '/mnt/Lavoro/halium'
It looks like my fstab was not complete, but missing the /boot partition. given that none of the fstab I located in my source tree had a line for the boot partition, I checked out the Android's fstab from the phone, via adb shell. There is no fstab in /etc, but two fstab files right in the root directory of the filesystem: fstab.sc8830 with exactly the same contents as the recovery fstab I found before in device/samsung/sharkls-common/recovery/root/fstab.sc8830, and a fstab.goldfish:
127|root@j3xlte:/ # cat /fstab.goldfish # Android fstab file. #<src> <mnt_point> <type> <mnt_flags and options> <fs_mgr_flags> # The filesystem that contains the filesystem checker binary (typically /system) cannot # specify MF_CHECK, and must come before any filesystems that do specify MF_CHECK /dev/block/mtdblock0 /system ext4 ro,noatime,barrier=1 wait /dev/block/mtdblock1 /data ext4 noatime,nosuid,nodev,barrier=1,nomblk_io_submit wait,check /dev/block/mtdblock2 /cache ext4 noatime,nosuid,nodev wait,check /devices/platform/goldfish_mmc.0 auto vfat defaults voldmanaged=sdcard:auto
While mount says:
~ # mount rootfs on / type rootfs (rw,seclabel) tmpfs on /dev type tmpfs (rw,seclabel,nosuid,relatime,mode=755) devpts on /dev/pts type devpts (rw,seclabel,relatime,mode=600) proc on /proc type proc (rw,relatime,gid=3009,hidepid=2) sysfs on /sys type sysfs (rw,seclabel,relatime) selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime) tmpfs on /tmp type tmpfs (rw,seclabel,relatime) pstore on /sys/fs/pstore type pstore (rw,seclabel,relatime) adb on /dev/usb-ffs/adb type functionfs (rw,relatime) adb on /dev/usb-ffs/adb type functionfs (rw,relatime) /dev/block/mmcblk0p27 on /data type ext4 (rw,seclabel,relatime,data=ordered) /dev/block/mmcblk0p27 on /sdcard type ext4 (rw,seclabel,relatime,data=ordered) /dev/block/mmcblk0p24 on /cache type ext4 (rw,seclabel,relatime,data=ordered)
None of the above helps in locating the boot partition. I then tried to look under /dev/block:
root@j3xlte:/dev/block/platform/sdio_emmc/by-name # ls -l lrwxrwxrwx root root 2019-06-26 19:13 CACHE -> /dev/block/mmcblk0p24 lrwxrwxrwx root root 2019-06-26 19:13 FOTA_SIG -> /dev/block/mmcblk0p11 lrwxrwxrwx root root 2019-06-26 19:13 HIDDEN -> /dev/block/mmcblk0p26 lrwxrwxrwx root root 2019-06-26 19:13 KERNEL -> /dev/block/mmcblk0p20 lrwxrwxrwx root root 2019-06-26 19:13 PARAM -> /dev/block/mmcblk0p16 lrwxrwxrwx root root 2019-06-26 19:13 PERSDATA -> /dev/block/mmcblk0p23 lrwxrwxrwx root root 2019-06-26 19:13 PERSISTENT -> /dev/block/mmcblk0p22 lrwxrwxrwx root root 2019-06-26 19:13 RECOVERY -> /dev/block/mmcblk0p21 lrwxrwxrwx root root 2019-06-26 19:13 RESERVED2 -> /dev/block/mmcblk0p19 lrwxrwxrwx root root 2019-06-26 19:13 SBOOT -> /dev/block/mmcblk0p1 lrwxrwxrwx root root 2019-06-26 19:13 SBOOT2 -> /dev/block/mmcblk0p2 lrwxrwxrwx root root 2019-06-26 19:13 SYSTEM -> /dev/block/mmcblk0p25 lrwxrwxrwx root root 2019-06-26 19:13 efs -> /dev/block/mmcblk0p17 lrwxrwxrwx root root 2019-06-26 19:13 l_fixnv1 -> /dev/block/mmcblk0p3 lrwxrwxrwx root root 2019-06-26 19:13 l_fixnv2 -> /dev/block/mmcblk0p4 lrwxrwxrwx root root 2019-06-26 19:13 l_gdsp -> /dev/block/mmcblk0p9 lrwxrwxrwx root root 2019-06-26 19:13 l_ldsp -> /dev/block/mmcblk0p7 lrwxrwxrwx root root 2019-06-26 19:13 l_modem -> /dev/block/mmcblk0p8 lrwxrwxrwx root root 2019-06-26 19:13 l_runtimenv1 -> /dev/block/mmcblk0p12 lrwxrwxrwx root root 2019-06-26 19:13 l_runtimenv2 -> /dev/block/mmcblk0p13 lrwxrwxrwx root root 2019-06-26 19:13 l_warm -> /dev/block/mmcblk0p10 lrwxrwxrwx root root 2019-06-26 19:13 pm_sys -> /dev/block/mmcblk0p5 lrwxrwxrwx root root 2019-06-26 19:13 prodnv -> /dev/block/mmcblk0p18 lrwxrwxrwx root root 2019-06-26 19:13 rsvdfixnv1 -> /dev/block/mmcblk0p6 lrwxrwxrwx root root 2019-06-26 19:13 td_runtimenv1 -> /dev/block/mmcblk0p14 lrwxrwxrwx root root 2019-06-26 19:13 td_runtimenv2 -> /dev/block/mmcblk0p15 lrwxrwxrwx root root 2019-06-26 19:13 userdata -> /dev/block/mmcblk0p27
This, combined from the PIT file which we can extract with Heimdall from the "Odin mode", gives us a better understanding of the device partitions:
$ sudo heimdall print-pit --no-reboot Heimdall v1.4.2 Copyright (c) 2010-2017 Benjamin Dobell, Glass Echidna http://www.glassechidna.com.au/ This software is provided free of charge. Copying and redistribution is encouraged. If you appreciate this software and you would like to support future development please consider donating: http://www.glassechidna.com.au/donate/ Initialising connection... Detecting device... Claiming interface... Setting up interface... Initialising protocol... Protocol initialisation successful. Beginning session... Some devices may take up to 2 minutes to respond. Please be patient! Session begun. Downloading device's PIT file... PIT file download successful. Entry Count: 31 Unknown 1: 1598902083 Unknown 2: 844251476 Unknown 3: 20563 Unknown 4: 17490 Unknown 5: 14136 Unknown 6: 13619 Unknown 7: 0 Unknown 8: 0 --- Entry #0 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 80 Attributes: 2 (STL Read-Only) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 0 Partition Block Count: 1024 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: BOOT Flash Filename: spl.img FOTA Filename: --- Entry #1 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 90 Attributes: 2 (STL Read-Only) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 0 Partition Block Count: 2048 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: BOOT2 Flash Filename: spl2.img FOTA Filename: --- Entry #2 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 70 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 1024 Partition Block Count: 1024 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: PIT Flash Filename: J3XLTE.pit FOTA Filename: --- Entry #3 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 71 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 2048 Partition Block Count: 6144 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: MD5HDR Flash Filename: md5.img FOTA Filename: --- Entry #4 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 1 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 8192 Partition Block Count: 4096 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: SBOOT Flash Filename: sboot.bin FOTA Filename: --- Entry #5 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 2 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 12288 Partition Block Count: 4096 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: SBOOT2 Flash Filename: sboot2.bin FOTA Filename: --- Entry #6 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 3 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 16384 Partition Block Count: 2048 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: l_fixnv1 Flash Filename: nvitem1.bin FOTA Filename: --- Entry #7 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 4 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 18432 Partition Block Count: 2048 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: l_fixnv2 Flash Filename: nvitem.bin FOTA Filename: --- Entry #8 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 5 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 20480 Partition Block Count: 2048 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: pm_sys Flash Filename: PM_sharkl_arm7.bin FOTA Filename: --- Entry #9 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 6 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 22528 Partition Block Count: 2048 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: rsvdfixnv1 Flash Filename: FOTA Filename: --- Entry #10 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 7 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 24576 Partition Block Count: 8192 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: l_ldsp Flash Filename: SPRDLTEDSP.img FOTA Filename: --- Entry #11 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 8 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 32768 Partition Block Count: 32768 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: l_modem Flash Filename: SPRDCP.img FOTA Filename: --- Entry #12 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 9 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 65536 Partition Block Count: 8192 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: l_gdsp Flash Filename: SPRDGDSP.img FOTA Filename: --- Entry #13 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 10 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 73728 Partition Block Count: 8192 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: l_warm Flash Filename: SPRDWDSP.img FOTA Filename: --- Entry #14 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 11 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 81920 Partition Block Count: 2048 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: FOTA_SIG Flash Filename: FOTA Filename: --- Entry #15 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 12 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 83968 Partition Block Count: 2048 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: l_runtimenv1 Flash Filename: FOTA Filename: --- Entry #16 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 13 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 86016 Partition Block Count: 2048 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: l_runtimenv2 Flash Filename: FOTA Filename: --- Entry #17 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 14 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 88064 Partition Block Count: 2048 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: td_runtimenv1 Flash Filename: FOTA Filename: --- Entry #18 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 15 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 90112 Partition Block Count: 2048 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: td_runtimenv2 Flash Filename: FOTA Filename: --- Entry #19 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 16 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 92160 Partition Block Count: 4096 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: PARAM Flash Filename: param.lfs FOTA Filename: --- Entry #20 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 17 Attributes: 5 (Read/Write) Update Attributes: 5 (FOTA) Partition Block Size/Offset: 96256 Partition Block Count: 40960 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: efs Flash Filename: efs.img FOTA Filename: --- Entry #21 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 18 Attributes: 5 (Read/Write) Update Attributes: 5 (FOTA) Partition Block Size/Offset: 137216 Partition Block Count: 10240 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: prodnv Flash Filename: prodnv.img FOTA Filename: --- Entry #22 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 19 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 147456 Partition Block Count: 12288 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: RESERVED2 Flash Filename: FOTA Filename: --- Entry #23 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 20 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 159744 Partition Block Count: 40960 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: KERNEL Flash Filename: boot.img FOTA Filename: --- Entry #24 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 21 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 200704 Partition Block Count: 40960 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: RECOVERY Flash Filename: recovery.img FOTA Filename: --- Entry #25 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 22 Attributes: 5 (Read/Write) Update Attributes: 1 (FOTA) Partition Block Size/Offset: 241664 Partition Block Count: 2048 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: PERSISTENT Flash Filename: FOTA Filename: --- Entry #26 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 23 Attributes: 5 (Read/Write) Update Attributes: 5 (FOTA) Partition Block Size/Offset: 243712 Partition Block Count: 18432 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: PERSDATA Flash Filename: persdata.img FOTA Filename: --- Entry #27 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 24 Attributes: 5 (Read/Write) Update Attributes: 5 (FOTA) Partition Block Size/Offset: 262144 Partition Block Count: 409600 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: CACHE Flash Filename: cache.img FOTA Filename: --- Entry #28 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 25 Attributes: 5 (Read/Write) Update Attributes: 5 (FOTA) Partition Block Size/Offset: 671744 Partition Block Count: 4194304 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: SYSTEM Flash Filename: system.img FOTA Filename: --- Entry #29 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 26 Attributes: 5 (Read/Write) Update Attributes: 5 (FOTA) Partition Block Size/Offset: 4866048 Partition Block Count: 81920 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: HIDDEN Flash Filename: hidden.img FOTA Filename: --- Entry #30 --- Binary Type: 0 (AP) Device Type: 2 (MMC) Identifier: 27 Attributes: 5 (Read/Write) Update Attributes: 5 (FOTA) Partition Block Size/Offset: 4947968 Partition Block Count: 0 File Offset (Obsolete): 0 File Size (Obsolete): 0 Partition Name: userdata Flash Filename: userdata.img FOTA Filename: remained Ending session... Releasing device interface...
Looking at the name, size and flash filename of the partitions, we can guess that the BOOT and BOOT2 are about splash screens, and what we need to use as our boot partition is in fact the KERNEL partition: /dev/block/mmcblk0p20 (note how the Identifier field from the PIT file matches what we got from block devices in the /dev filesystem). So, we edit our fstab file and add a line
/dev/block/platform/sdio_emmc/by-name/KERNEL /boot ext4 ro wait
From the IRC #halium channel, I understood that the fixup mountpoints step is needed. The block I added to the halium/hybris-boot/fixup-mountpoints script is this:
"j3xlte")
sed -i \
-e 's block/platform/sdio_emmc/by-name/SYSTEM mmcblk0p25 ' \
-e 's block/platform/sdio_emmc/by-name/userdata mmcblk0p27 ' \
-e 's block/platform/sdio_emmc/by-name/CACHE mmcblk0p24 ' \
-e 's block/platform/sdio_emmc/by-name/efs mmcblk0p17 ' \
-e 's block/platform/sdio_emmc/by-name/prodnv mmcblk0p18 ' \
-e 's block/platform/sdio_emmc/by-name/KERNEL mmcblk0p20 ' \
"$@"
;;
I then ran mka halium-boot again, and it finally started to build the kernel. But it soon stopped:
/mnt/Lavoro/halium/kernel/samsung/sharkls/kernel/cgroup.c: In function 'subsys_cgroup_allow_attach':
/mnt/Lavoro/halium/kernel/samsung/sharkls/kernel/cgroup.c:2138:37: error: invalid operands to binary != (have 'kuid_t' and 'kuid_t')
if (current != task && cred->euid != tcred->uid &&
^
/mnt/Lavoro/halium/kernel/samsung/sharkls/kernel/cgroup.c:2139:18: error: invalid operands to binary != (have 'kuid_t' and 'kuid_t')
cred->euid != tcred->suid)
^
Luckily, this is one of the known build errors, so I didn't waste a lot of time on it: setting CONFIG_USER_NS=n in kernel/samsung/sharkls/arch/arm/configs/j3xlte_defconfig fixed it. Then the build failed with
LD init/built-in.o mm/built-in.o: In function `set_AKSM_level': /mnt/Lavoro/halium/kernel/samsung/sharkls/mm/ksm.c:1822: undefined reference to `get_minfree_high_value' /mnt/Lavoro/halium/kernel/samsung/sharkls/Makefile:786: recipe for target 'vmlinux' failed make[1]: *** [vmlinux] Error 1 Makefile:130: recipe for target 'sub-make' failed make: *** [sub-make] Error 2
Looking at the code, I understood I could probably switch off this code path with CONFIG_ADAPTIVE_KSM=n. After doing that, got another error:
LD init/built-in.o arch/arm/lib/lib.a(memcmpksm.o):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr0' /mnt/Lavoro/halium/kernel/samsung/sharkls/Makefile:786: recipe for target 'vmlinux' failed make[1]: *** [vmlinux] Error 1 Makefile:130: recipe for target 'sub-make' failed make: *** [sub-make] Error 2
Disabling also CONFIG_KSM_ASSEMBLY_MEMCMP did the trick:
CC /mnt/Lavoro/halium/vendor/sprd/wcn/wifi/sc2331/6.0/sprdwl.mod.o LD [M] /mnt/Lavoro/halium/vendor/sprd/wcn/wifi/sc2331/6.0/sprdwl.ko make[1]: Leaving directory '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/KERNEL_OBJ' make: Leaving directory '/mnt/Lavoro/halium/vendor/sprd/wcn/wifi/sc2331/6.0' [ 77% 7/9] Target dt image: /mnt/Lavoro/halium/out/target/product/j3xlte/dt.img DTB combiner: Input directory: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/KERNEL_OBJ/arch/arm/boot/dts/' Output file: '/mnt/Lavoro/halium/out/target/product/j3xlte/dt.img' Found file: sprd-scx35l_sharkls_j3xlte_rev03.dtb ... chipset: 9830, platform: 3, rev: 131072 Found file: sprd-scx35l_sharkls_j3xlte_rev02.dtb ... chipset: 9830, platform: 2, rev: 131072 Found file: sprd-scx35l_sharkls_j3xlte_rev06.dtb ... chipset: 9830, platform: 6, rev: 131072 Found file: sprd-scx35l_sharkls_j3xlte_rev07.dtb ... chipset: 9830, platform: 7, rev: 131072 Found file: sprd-scx35l_sharkls_j3xlte_rev04.dtb ... chipset: 9830, platform: 4, rev: 131072 Found file: sprd-scx35l_sharkls_j3xlte_rev01.dtb ... chipset: 9830, platform: 1, rev: 131072 Found file: sprd-scx35l_sharkls_j3xlte_rev05.dtb ... chipset: 9830, platform: 5, rev: 131072 Found file: sprd-scx35l_sharkls_j3xlte_rev00.dtb ... chipset: 9830, platform: 0, rev: 131072 => Found 8 unique DTB(s) Generating master DTB... completed Made DT image: /mnt/Lavoro/halium/out/target/product/j3xlte/dt.img [100% 9/9] Install: /mnt/Lavoro/halium/out/target/product/j3xlte/halium-boot.img make: Leaving directory '/mnt/Lavoro/halium' #### make completed successfully (01:28 (mm:ss)) ####
Building the Halium system image
Continuing following the Halium documentation, the next step was building the system image with the mka systemimage command. Of course, it failed:
make: Entering directory '/mnt/Lavoro/halium' ============================================ PLATFORM_VERSION_CODENAME=REL PLATFORM_VERSION=7.1.1 LINEAGE_VERSION=14.1-20190702-UNOFFICIAL-j3xlte TARGET_PRODUCT=lineage_j3xlte TARGET_BUILD_VARIANT=userdebug TARGET_BUILD_TYPE=release TARGET_BUILD_APPS= TARGET_ARCH=arm TARGET_ARCH_VARIANT=armv7-a-neon TARGET_CPU_VARIANT=cortex-a7 TARGET_2ND_ARCH= TARGET_2ND_ARCH_VARIANT= TARGET_2ND_CPU_VARIANT= HOST_ARCH=x86_64 HOST_2ND_ARCH=x86 HOST_OS=linux HOST_OS_EXTRA=Linux-4.15.0-52-generic-x86_64-with-Ubuntu-16.04-xenial HOST_CROSS_OS=windows HOST_CROSS_ARCH=x86 HOST_CROSS_2ND_ARCH=x86_64 HOST_BUILD_TYPE=release BUILD_ID=NOF27B OUT_DIR=/mnt/Lavoro/halium/out RECOVERY_VARIANT=twrp WITH_SU=true ============================================ Running kati to generate build-lineage_j3xlte.ninja... No need to regenerate ninja file Starting build with ninja ninja: Entering directory `.' ninja: error: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libcurl_intermediates/export_includes', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstrongswan_intermediates/import_includes', missing and no known rule to make it build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed make: *** [ninja_wrapper] Error 1 make: Leaving directory '/mnt/Lavoro/halium' #### make failed to build some targets (2 seconds) ####
I found similar (though not exactly the same) errors in #halium: the solution is to locate the .mk file causing the build of the package requiring the missing file, and remove it. I ran a repo grep libstrongswan, and found that it's mentioned in device/samsung/sharkls-common/sharkls.mk. I removed it from there, but I got the same build error. Removing the whole block of PRODUCT_PACKAGES led to some progress:
build/core/binary.mk:1253: vendor/sprd/modules/libcamera/Android.mk: libcamisp2.0: Unused source files: isp2.0/calibration/backup isp2.0/third_lib/alc_ip build/core/binary.mk:1253: vendor/sprd/open-source/libs/vpu/mmf/openmax/libomxil-bellagio-0.9.3/Android.mk: omxregister-bellagio: Unused source files: src/omxregister.h No private recovery resources for TARGET_DEVICE j3xlte build/core/Makefile:34: warning: overriding commands for target `/mnt/Lavoro/halium/out/target/product/j3xlte/system/etc/wifi/wpa_supplicant.conf' build/core/base_rules.mk:316: warning: ignoring old commands for target `/mnt/Lavoro/halium/out/target/product/j3xlte/system/etc/wifi/wpa_supplicant.conf' Starting build with ninja ninja: Entering directory `.' ninja: error: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/STATIC_LIBRARIES/libbt-utils_intermediates/export_includes', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/EXECUTABLES/engpc_intermediates/import_includes', missing and no known rule to make it build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed make: *** [ninja_wrapper] Error 1
I then removed engpc from the same sharkls.mk file:
Starting build with ninja ninja: Entering directory `.' ninja: error: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/STATIC_LIBRARIES/libbootloader_message_intermediates/export_includes', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/EXECUTABLES/vold_intermediates/import_includes', missing and no known rule to make it build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed make: *** [ninja_wrapper] Error 1
The last comment from here suggests that vold might not be needed (though some people in #halium claimed that it could be needed, and indeed in the Meizu Pro 5 by Canonical I do have that program under /system/bin/). Anyway, I edited build/target/product/base.mk and removed vold from the PRODUCT_PACKAGES variable. I also read that it's recommended to run the make command as LANG=C mka systemimage, since the outcome can be affected by the user locale. Next:
Starting build with ninja ninja: Entering directory `.' ninja: error: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libfmjni_intermediates/export_includes', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengfm_intermediates/import_includes', missing and no known rule to make it build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed make: *** [ninja_wrapper] Error 1
This is mentioned in vendor/sprd/wcn/platform/wcnd/Android.mk, so let's remove it from there (LOCAL_SHARED_LIBRARIES variable):
Starting build with ninja ninja: Entering directory `.' ninja: error: 'vendor/samsung/j3xlte/proprietary/bin/at_distributor', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/system/bin/at_distributor', missing and no known rule to make it build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed make: *** [ninja_wrapper] Error 1
This is brought in by device/samsung/j3xlte/proprietary-files.txt, so I commented it out from there. This didn't seem to make any difference, and a mka clean didn't help either. Turns out it's also mentioned in vendor/samsung/j3xlte/j3xlte-vendor-blobs.mk, so I removed it from there too (PRODUCT_COPY_FILES variable). Reading the comment block on top of this makefile we understand that it's generated by the setup-makefiles.sh script based on the contents of proprietary-files.txt. So, one option is to re-run the setup-makefiles.sh script, but I preferred to take a no risk approach and edit both files by hand. The next failure was about mfgloader, which is also mentioned in the same file:
Starting build with ninja ninja: Entering directory `.' ninja: error: 'vendor/samsung/j3xlte/proprietary/bin/mfgloader', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/system/bin/mfgloader', missing and no known rule to make it build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed make: *** [ninja_wrapper] Error 1
Let's try to remove it as well, from the same two files:
Starting build with ninja ninja: Entering directory `.' ninja: error: 'vendor/samsung/j3xlte/proprietary/bin/sprdSleepLog', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/system/bin/sprdSleepLog', missing and no known rule to make it build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed make: *** [ninja_wrapper] Error 1
Next is modemd (exactly same logs). At this point I started suspecting that this was not the right way to proceed, that that instead I was just missing all the proprietary blobs. The device tree repository has an extract-files.sh which seems to get these binaries from a device. After looking at its source, it looks like it must be run from the device/samsung/j3xlte directory with the device connected and reachable via adb. Before running it, I reverted my changes to the proprietary-files.txt file, in order to retrieve all the needed blobs. Running the extraction tool failed once:
./extract-files.sh 846 KB/s (92260 bytes in 0.106s) 108 KB/s (9664 bytes in 0.086s) 106 KB/s (9444 bytes in 0.086s) 279 KB/s (34092 bytes in 0.119s) 44 KB/s (1992 bytes in 0.043s) 115 KB/s (5296 bytes in 0.044s) 2 KB/s (265 bytes in 0.087s) 40 KB/s (3770 bytes in 0.090s) 25 KB/s (2247 bytes in 0.087s) 4 KB/s (427 bytes in 0.087s) 0 KB/s (70 bytes in 0.087s) remote object '/system/bin/IPSecService' does not exist
but after removing the missing file from proprietary-files.txt, the extraction completed successfully. However, one file was still not retrieved, as this line from the output showed:
[...] 4779 KB/s (1386252 bytes in 0.283s) 405 KB/s (37928 bytes in 0.091s) failed to copy '/system/vendor/firmware/vbc_eq' to '../../../vendor/samsung/j3xlte/proprietary/vendor/firmware/vbc_eq': Permission denied 3015 KB/s (415875 bytes in 0.134s) [...]
I checked the situation by connecting to the device with adb shell, and the file was there but readable by root only. After copying the file to /data/user and calling chmod 0666 vbc_eq, I could manually retrieve it with
adb pull /data/user/vbc_eq ../../../vendor/samsung/j3xlte/proprietary/vendor/firmware/vbc_eq
I preferred not to touch the permissions directly on the original file, not to risk messing up with the system; and after retrieving the file, I removed my copy from the device. The I ran
JOBS=2 ./halium/devices/setup j3xlte
once again. That completed fine, and the next mka systemimage failed with another error:
Starting build with ninja ninja: Entering directory `.' ninja: error: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/STATIC_LIBRARIES/libbootloader_message_intermediates/export_includes', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/EXECUTABLES/init_intermediates/import_includes', missing and no known rule to make it build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed make: *** [ninja_wrapper] Error 1
I played with the android_system_core repository, which is checked out in system/core/, to remove references to bootloader_message from init/builtins.cpp, and I also removed it from the LOCAL_STATIC_LIBRARIES variable in the system/core/init/Android.mk file. Next:
Starting build with ninja ninja: Entering directory `.' ninja: error: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/STATIC_LIBRARIES/libminui_intermediates/export_includes', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/EXECUTABLES/healthd_intermediates/import_includes', missing and no known rule to make it build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed make: *** [ninja_wrapper] Error 1
This is listed in build/target/product/core_base.mk and build/target/product/embedded.mk and needs to be removed from both (it was disabled in Halium's init, so it's definitely not needed). It then failed with:
Starting build with ninja ninja: Entering directory `.' ninja: error: 'bootable/recovery-twrp/etc/init.rc', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/ramdisk-recovery.cpio', missing and no known rule to make it build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed make: *** [ninja_wrapper] Error 1
The recovery-twrp project is listed in device/samsung/sharkls-common/lineage.dependencies, so let's try adding a line
<project path="bootable/recovery-twrp" name="omnirom/android_bootable_recovery" remote="github" revision="android-7.1" />
to our manifest file and run JOBS=2 ./halium/devices/setup j3xlte once more. After that, mka systemimage gives another error:
Starting build with ninja ninja: Entering directory `.' ninja: error: '/mnt/Lavoro/halium/out/host/linux-x86/bin/imgdiff', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/recovery_patch_intermediates/recovery_from_boot.p', missing and no known rule to make it build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed make: *** [ninja_wrapper] Error 1
Adding bootable/recovery-twrp to the subdirs variable in build/core/main.mk seems to get us further. But before showing you the next error, I need to stop for a second, because at this point I started suspecting that the previous error about the bootloader_message library could go away now that I started using the same recovery-twrp tree used by the LineageOS porter. So I reverted my changes about the bootloader_message library and indeed it looked like the previous error was gone forever. And it might be that some of the other stuff I disabled above might be restored back; but I decided to leave it for later (it's documented in this blog post, after all, so I can go back to it at any time) and proceed with the build:
Starting build with ninja
ninja: Entering directory `.'
[ 0% 125/16364] host C++: libadb <= system/core/adb/usb_linux.cpp
FAILED: /bin/bash -c "(PWD=/proc/self/cwd prebuilts/clang/host/linux-x86/clang-2690385/bin/clang++ -I device/samsung/sharkls-common/include -I system/core/adb -I /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates -I /mnt/Lavoro/halium/out/host/linux-x86/gen/STATIC_LIBRARIES/libadb_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem tools/include -isystem /mnt/Lavoro/halium/out/host/linux-x86/obj/include -c -fno-exceptions -Wno-multichar -m64 -Wa,--noexecstack -fPIC -no-canonical-prefixes -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector -D__STDC_FORMAT_MACROS -D__STDC_CONSTANT_MACROS -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -O2 -g -fno-strict-aliasing -DNDEBUG -UDEBUG -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics --gcc-toolchain=prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8 -fstack-protector-strong --gcc-toolchain=prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8 --sysroot prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/sysroot -Bprebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/x86_64-linux/bin -target x86_64-linux-gnu -Wsign-promo -Wno-inconsistent-missing-override --gcc-toolchain=prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8 --sysroot prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/sysroot -isystem prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/x86_64-linux/include/c++/4.8 -isystem prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/x86_64-linux/include/c++/4.8/x86_64-linux -isystem prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/x86_64-linux/include/c++/4.8/backward -target x86_64-linux-gnu -Wall -Wextra -Werror -Wno-unused-parameter -Wno-missing-field-initializers -Wvla -DADB_REVISION='\"80811505f1ff-android\"' -fvisibility=hidden -DADB_HOST=1 -fPIC -std=c++14 -Wexit-time-destructors -D_USING_LIBCXX -std=gnu++14 -nostdinc++ -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type -MD -MF /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.d -o /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.o system/core/adb/usb_linux.cpp ) && (cp /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.d /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.d >> /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.P; rm -f /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.d )"
In file included from system/core/adb/usb_linux.cpp:25:
In file included from device/samsung/sharkls-common/include/linux/usb/ch9.h:35:
In file included from device/samsung/sharkls-common/include/uapi/linux/usb/ch9.h:36:
In file included from device/samsung/sharkls-common/include/linux/types.h:5:
In file included from device/samsung/sharkls-common/include/uapi/linux/types.h:13:
In file included from prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/sysroot/usr/include/linux/posix_types.h:4:
device/samsung/sharkls-common/include/linux/stddef.h:11:2: error: expected identifier
false = 0,
^
prebuilts/clang/host/linux-x86/clang-2690385/bin/../lib64/clang/3.8.256229/include/stdbool.h:38:15: note: expanded from macro 'false'
#define false false
^
(see here for the full log). After some resultless internet search and asking around, I tried to look into the issue myself, and noticed that in that failed command there's a mix of headers coming from the userspace and from the kernel (which usually are not used together). Since we are building userspace programs now, it seems a bit strange that there are also kernel headers being used (those under device/samsung/sharkls-common). Looking at system/core/adb/usb_linux.cpp, I saw that it contains a line #include <linux/usb/ch9.h> which causes the inclusion of device/samsung/sharkls-common/include/linux/usb/ch9.h; but maybe there are other ch9.h files in the build tree, provided by userspace libraries and not by the kernel? Indeed, a find showed that I had also a ./prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/sysroot/usr/include/linux/usb/ch9.h file, which should probably be used instead of the kernel's one. So I copied the failing command line (starting from what comes after FAILED:) into a terminal and ran it, and could reproduce the issue; but if after removing the first include directive (-I device/samsung/sharkls-common/include) the command succeeded.
The line is added in build/core/binary.mk:
my_c_includes := $(TOPDIR)$(TARGET_SPECIFIC_HEADER_PATH) $(my_c_includes)
(I found it out by modifying the file, and adding random text strings around the right-hand parts of these assignments, then running the build again and see which random strings were surrounding the device/samsung/sharkls-common/include text in the failing command line). So, we need to find why TARGET_SPECIFIC_HEADER_PATH is set. I ran a repo grep TARGET_SPECIFIC_HEADER_PATH and found that it's being set in device/samsung/sharkls-common/BoardConfigCommon.mk. Browsing the various directory trees, and double-checking the manifest file, I realized that the device/samsung/sharkls-common should not contain the kernel source tree, yet in my local disk it did (and all the files were untracked by git). It was probably the result of one of my previous attempts of compiling the manifest file, where some projects were checked out locally but I didn't clean them up after updating the manifest. I therefore ran a git clean -dfx in that directory, and the kernel source were gone. The next build continued for a while, then failed:
[ 76% 12454/16364] target thumb C: libengbt <= vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c
FAILED: /bin/bash -c "(PWD=/proc/self/cwd prebuilts/clang/host/linux-x86/clang-2690385/bin/clang -I device/samsung/sharkls-common/include -I system/bt/stack/include -I system/bt/include -I vendor/sprd/open-source/apps/engmode/bt -I /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates -I /mnt/Lavoro/halium/out/target/product/j3xlte/gen/SHARED_LIBRARIES/libengbt_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem hardware/ril/include -isystem /mnt/Lavoro/halium/out/target/product/j3xlte/obj/include -isystem bionic/libc/arch-arm/include -isystem bionic/libc/include -isystem bionic/libc/kernel/uapi -isystem bionic/libc/kernel/common -isystem bionic/libc/kernel/uapi/asm-arm -isystem bionic/libm/include -isystem bionic/libm/include/arm -c -fno-exceptions -Wno-multichar -msoft-float -ffunction-sections -fdata-sections -funwind-tables -fstack-protector-strong -Wa,--noexecstack -Werror=format-security -D_FORTIFY_SOURCE=2 -fno-short-enums -no-canonical-prefixes -mcpu=cortex-a7 -mfpu=neon-vfpv4 -D__ARM_FEATURE_LPAE=1 -mfloat-abi=softfp -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -DNDEBUG -g -Wstrict-aliasing=2 -DNDEBUG -UDEBUG -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -nostdlibinc -target arm-linux-androideabi -target arm-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin -std=gnu99 -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing -DHAS_NO_BDROID_BUILDCFG -DSPRD_WCNBT_MARLIN -fPIC -D_USING_LIBCXX -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type -MD -MF /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d -o /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.o vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c ) && (cp /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d >> /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.P; rm -f /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d )"
vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c:30:10: fatal error: 'bt_types.h' file not found
#include "bt_types.h"
^
1 error generated.
Indeed the bt_types.h was nowhere to be found in my source tree. A search in GitHub reveals that this file is provided by libncf-nci, so it's probably related to NFC. The local manifest from LineageOS indeed had a few lines with NFC-related projects (at least, judging from their name):
<project path="vendor/nxp-nfc/opensource/Nfc" name="LineageOS/android_vendor_nxp-nfc_opensource_Nfc" revision="cm-14.1" /> <project path="vendor/nxp-nfc/opensource/frameworks" name="LineageOS/android_vendor_nxp-nfc_opensource_frameworks" revision="cm-14.1" /> <project path="vendor/nxp-nfc/opensource/libnfc-nci" name="LineageOS/android_vendor_nxp-nfc_opensource_libnfc-nci" revision="cm-14.1" />
I initially excluded them from my manifest, given that my device doesn't have NFC, but at this point I thought I'd better include them, just for the sake of building the binaries (the alternative being removing the binaries required these libraries, or hacking on them to remove the dependency; this is a decision I might want to revisit later). So I added them to my manifest file (adding the proper remote attribute!), re-ran the setup script which fetched the newly-added repositories, but the build still failed with the same error, as the include lines were not updated. However, adding vendor/nxp-nfc/opensource/libnfc-nci/src/include to the TARGET_SPECIFIC_HEADER_PATH variable in device/samsung/sharkls-common/BoardConfigCommon.mk caused the build to fail even earlier. I then tried to add it to the local project being built (engmode), by changing the LOCAL_C_INCLUDES variable in vendor/sprd/open-source/apps/engmode/bt/Android.mk and that brought me just a little further:
[ 97% 1507/1552] target thumb C: libengbt <= vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c
FAILED: /bin/bash -c "(PWD=/proc/self/cwd prebuilts/clang/host/linux-x86/clang-2690385/bin/clang -I device/samsung/sharkls-common/include -I system/bt/stack/include -I vendor/nxp-nfc/opensource/libnfc-nci/src/include -I system/bt/include -I vendor/sprd/open-source/apps/engmode/bt -I /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates -I /mnt/Lavoro/halium/out/target/product/j3xlte/gen/SHARED_LIBRARIES/libengbt_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem hardware/ril/include -isystem /mnt/Lavoro/halium/out/target/product/j3xlte/obj/include -isystem bionic/libc/arch-arm/include -isystem bionic/libc/include -isystem bionic/libc/kernel/uapi -isystem bionic/libc/kernel/common -isystem bionic/libc/kernel/uapi/asm-arm -isystem bionic/libm/include -isystem bionic/libm/include/arm -c -fno-exceptions -Wno-multichar -msoft-float -ffunction-sections -fdata-sections -funwind-tables -fstack-protector-strong -Wa,--noexecstack -Werror=format-security -D_FORTIFY_SOURCE=2 -fno-short-enums -no-canonical-prefixes -mcpu=cortex-a7 -mfpu=neon-vfpv4 -D__ARM_FEATURE_LPAE=1 -mfloat-abi=softfp -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -DNDEBUG -g -Wstrict-aliasing=2 -DNDEBUG -UDEBUG -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -nostdlibinc -target arm-linux-androideabi -target arm-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin -std=gnu99 -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing -DHAS_NO_BDROID_BUILDCFG -DSPRD_WCNBT_MARLIN -fPIC -D_USING_LIBCXX -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type -MD -MF /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d -o /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.o vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c ) && (cp /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d >> /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.P; rm -f /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d )"
In file included from vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c:30:
vendor/nxp-nfc/opensource/libnfc-nci/src/include/bt_types.h:22:10: fatal error: 'data_types.h' file not found
#include "data_types.h"
^
1 error generated.
(note that in order to speed up the development I was no longer running mka systemimage, but mka libengbt, which is the library affected by the failure). I had three candidates:
vendor/nxp-nfc/opensource/libnfc-nci/p61-jcop-kit/inc/data_types.h vendor/nxp-nfc/opensource/libnfc-nci/src/gki/ulinux/data_types.h vendor/nxp-nfc/opensource/libnfc-nci/halimpl/bcm2079x/gki/ulinux/data_types.h
they didn't seem to be very different, so I picked the first one. I got then another failure:
[ 35% 12/34] target thumb C: libengbt <= vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c
FAILED: /bin/bash -c "(PWD=/proc/self/cwd prebuilts/clang/host/linux-x86/clang-2690385/bin/clang -I device/samsung/sharkls-common/include -I system/bt/stack/include -I vendor/nxp-nfc/opensource/libnfc-nci/src/include -I vendor/nxp-nfc/opensource/libnfc-nci/p61-jcop-kit/inc/ -I system/bt/include -I vendor/sprd/open-source/apps/engmode/bt -I /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates -I /mnt/Lavoro/halium/out/target/product/j3xlte/gen/SHARED_LIBRARIES/libengbt_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem hardware/ril/include -isystem /mnt/Lavoro/halium/out/target/product/j3xlte/obj/include -isystem bionic/libc/arch-arm/include -isystem bionic/libc/include -isystem bionic/libc/kernel/uapi -isystem bionic/libc/kernel/common -isystem bionic/libc/kernel/uapi/asm-arm -isystem bionic/libm/include -isystem bionic/libm/include/arm -c -fno-exceptions -Wno-multichar -msoft-float -ffunction-sections -fdata-sections -funwind-tables -fstack-protector-strong -Wa,--noexecstack -Werror=format-security -D_FORTIFY_SOURCE=2 -fno-short-enums -no-canonical-prefixes -mcpu=cortex-a7 -mfpu=neon-vfpv4 -D__ARM_FEATURE_LPAE=1 -mfloat-abi=softfp -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -DNDEBUG -g -Wstrict-aliasing=2 -DNDEBUG -UDEBUG -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -nostdlibinc -target arm-linux-androideabi -target arm-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin -std=gnu99 -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing -DHAS_NO_BDROID_BUILDCFG -DSPRD_WCNBT_MARLIN -fPIC -D_USING_LIBCXX -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type -MD -MF /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d -o /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.o vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c ) && (cp /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d >> /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.P; rm -f /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d )"
vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c:31:10: fatal error: 'btm_api.h' file not found
#include "btm_api.h"
^
1 error generated.
I search in the internet revealed that this is part of Bluedroid, which I assume is the Android bluetooth stack; so I looked in the Halium manifest directory for a repo line:
$ git grep blue motorola_potter.xml: <project path="external/bluetooth/bluedroid" name="android_external_bluetooth_bluedroid" remote="los" revision="cm-12.1" />
This one seemed promising, so I added it to my manifest. I ran the setup script again, and had a look at the newly downloaded external/bluetooth/bluedroid/stack/include/ directory: that not only included the btm_api.h file, but also the bt_types.h which we were needing before. So I went back and tried removing the two include lines related to the nxp-nfc projects and replaced them with this one from bluedroid. I got the same issue about the missing data_types.h file, so I added back the vendor/nxp-nfc/opensource/libnfc-nci/p61-jcop-kit/inc line. I then got an error about a missing tb_target.h file, which is found in external/bluetooth/bluedroid/include/, so I added this line as well. This let me build libengbt successfully. The systemimage target instead failed later:
[ 12% 502/3912] target thumb C++: audio.primary.sc8830 <= vendor/sprd/modules/audio/normal/audiotest/auto_audio_v2.cpp
FAILED: /bin/bash -c "(PWD=/proc/self/cwd prebuilts/clang/host/linux-x86/clang-2690385/bin/clang++ -I device/samsung/sharkls-common/include -I external/tinyalsa/include -I external/expat/lib -I system/media/audio_utils/include -I system/media/audio_effects/include -I vendor/sprd/open-source/apps/engmode -I vendor/sprd/modules/audio/normal/vb_pga -I vendor/sprd/modules/audio/normal/record_process -I vendor/sprd/modules/audio/normal/include -I vendor/sprd/modules/audio/normal/DumpData -I vendor/sprd/modules/audio/normal/audiotest -I vendor/sprd/modules/audio/normal/record_nr -I vendor/sprd/modules/audio/normal/skd -I vendor/sprd/modules/audio/normal/resample_api -I vendor/sprd/modules/audio/normal/custom_mmi -I vendor/sprd/modules/resampler -I vendor/sprd/modules/audio/normal/libaudioril -I vendor/sprd/open-source/libs/libatci -I vendor/sprd/modules/audio/normal/vb_effect/v2 -I vendor/sprd/modules/audio/normal -I /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates -I /mnt/Lavoro/halium/out/target/product/j3xlte/gen/SHARED_LIBRARIES/audio.primary.sc8830_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem hardware/ril/include -isystem /mnt/Lavoro/halium/out/target/product/j3xlte/obj/include -isystem bionic/libc/arch-arm/include -isystem bionic/libc/include -isystem bionic/libc/kernel/uapi -isystem bionic/libc/kernel/common -isystem bionic/libc/kernel/uapi/asm-arm -isystem bionic/libm/include -isystem bionic/libm/include/arm -c -fno-exceptions -Wno-multichar -msoft-float -ffunction-sections -fdata-sections -funwind-tables -fstack-protector-strong -Wa,--noexecstack -Werror=format-security -D_FORTIFY_SOURCE=2 -fno-short-enums -no-canonical-prefixes -mcpu=cortex-a7 -mfpu=neon-vfpv4 -D__ARM_FEATURE_LPAE=1 -mfloat-abi=softfp -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -DNDEBUG -g -Wstrict-aliasing=2 -DNDEBUG -UDEBUG -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -nostdlibinc -target arm-linux-androideabi -target arm-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin -fvisibility-inlines-hidden -Wsign-promo -Wno-inconsistent-missing-override -nostdlibinc -target arm-linux-androideabi -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing -fno-rtti -D_POSIX_SOURCE -Wno-multichar -g -DAUDIO_SPIPE_TD -D_LPA_IRAM -DVOIP_DSP_PROCESS -DAUDIO_HAL_ANDROID_N_API -DFM_VERSION_IS_GOOGLE -fPIC -D_USING_LIBCXX -std=gnu++14 -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type -MD -MF /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.d -o /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.o vendor/sprd/modules/audio/normal/audiotest/auto_audio_v2.cpp ) && (cp /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.d /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.d >> /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.P; rm -f /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.d )"
vendor/sprd/modules/audio/normal/audiotest/auto_audio_v2.cpp:21:9: warning: 'LOG_TAG' macro redefined [-Wmacro-redefined]
#define LOG_TAG "SPRD_AUDIOTRACK"
^
system/core/include/log/log.h:65:9: note: previous definition is here
#define LOG_TAG NULL
^
vendor/sprd/modules/audio/normal/audiotest/auto_audio_v2.cpp:186:39: warning: unused parameter 'user' [-Wunused-parameter]
void playerCallback( int event, void* user, void *info )
^
vendor/sprd/modules/audio/normal/audiotest/auto_audio_v2.cpp:245:9: error: use of undeclared identifier 'AUDIO_DEVICE_OUT_FM_HEADSET'; did you mean 'AUDIO_DEVICE_OUT_WIRED_HEADSET'?
AUDIO_DEVICE_OUT_FM_HEADSET,
^~~~~~~~~~~~~~~~~~~~~~~~~~~
AUDIO_DEVICE_OUT_WIRED_HEADSET
system/media/audio/include/system/audio.h:711:5: note: 'AUDIO_DEVICE_OUT_WIRED_HEADSET' declared here
AUDIO_DEVICE_OUT_WIRED_HEADSET = 0x4,
^
This is an issue reported here, The link mentioned there is for CyanogenMod 13, but I found one for LineageOS 14.1 here. So I ran
cd device/samsung/sharkls-common/patches/ ./apply_sprd-diff.sh
This failed on a couple of patches, for which I reported an issue here, though it's likely not important for us (as it mainly regards Java code). The build proceeded:
[ 10% 436/4011] target thumb C++: libstagefright <= frameworks/av/media/libstagefright/CameraSource.cpp
FAILED: /bin/bash -c "(PWD=/proc/self/cwd prebuilts/clang/host/linux-x86/clang-2690385/bin/clang++ -I device/samsung/sharkls-common/include -I ./frameworks/av/include/media/ -I ./frameworks/av/media/libavextensions -I ./frameworks/av/media/libstagefright/mpeg2ts -I ./frameworks/av/include/media/stagefright/timedtext -I ./frameworks/native/include/media/hardware -I ./frameworks/native/include/media/openmax -I ./external/flac/include -I ./external/tremolo -I ./external/libvpx/libwebm -I ./system/netd/include -I system/media/audio_utils/include -I hardware/qcom/media/sc8830/mm-core/inc -I ./external/stagefright-plugins/include -I frameworks/av/media/libstagefright -I /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates -I /mnt/Lavoro/halium/out/target/product/j3xlte/gen/SHARED_LIBRARIES/libstagefright_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem hardware/ril/include -isystem /mnt/Lavoro/halium/out/target/product/j3xlte/obj/include -isystem bionic/libc/arch-arm/include -isystem bionic/libc/include -isystem bionic/libc/kernel/uapi -isystem bionic/libc/kernel/common -isystem bionic/libc/kernel/uapi/asm-arm -isystem bionic/libm/include -isystem bionic/libm/include/arm -c -fno-exceptions -Wno-multichar -msoft-float -ffunction-sections -fdata-sections -funwind-tables -fstack-protector-strong -Wa,--noexecstack -Werror=format-security -D_FORTIFY_SOURCE=2 -fno-short-enums -no-canonical-prefixes -mcpu=cortex-a7 -mfpu=neon-vfpv4 -D__ARM_FEATURE_LPAE=1 -mfloat-abi=softfp -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -DNDEBUG -g -Wstrict-aliasing=2 -DNDEBUG -UDEBUG -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -nostdlibinc -target arm-linux-androideabi -target arm-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin -fvisibility-inlines-hidden -Wsign-promo -Wno-inconsistent-missing-override -nostdlibinc -target arm-linux-androideabi -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing -fno-rtti -Wno-multichar -Werror -Wno-error=deprecated-declarations -Wall -DENABLE_STAGEFRIGHT_EXPERIMENTS -DUSE_SPRD_COLORFORMAT -fPIC -D_USING_LIBCXX -fsanitize=unsigned-integer-overflow,signed-integer-overflow -fsanitize-trap=all -ftrap-function=abort -std=gnu++14 -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type -MD -MF /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.d -o /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.o frameworks/av/media/libstagefright/CameraSource.cpp ) && (cp /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.d /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.d >> /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.P; rm -f /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.d )"
frameworks/av/media/libstagefright/CameraSource.cpp:134:16: error: use of undeclared identifier 'OMX_SPRD_COLOR_FormatYVU420SemiPlanar'; did you mean 'OMX_QCOM_COLOR_FormatYVU420SemiPlanar'?
return OMX_SPRD_COLOR_FormatYVU420SemiPlanar;
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
OMX_QCOM_COLOR_FormatYVU420SemiPlanar
./frameworks/native/include/media/openmax/OMX_IVCommon.h:169:5: note: 'OMX_QCOM_COLOR_FormatYVU420SemiPlanar' declared here
OMX_QCOM_COLOR_FormatYVU420SemiPlanar = 0x7FA30C00,
^
1 error generated.
I realized that there were more patches to be applied, and that the script stopped as soon as it stumbled into a failing patch. So, I took all the failing patches out of the way:
mv sprd-diff/frameworks_base.diff{,.skip} mv sprd-diff/frameworks_opt.diff{,.skip} mv sprd-diff/packages_apps.diff{,.skip} mv sprd-diff/packages_services.diff{,.skip} mv sprd-diff/system_bt.diff{,.skip} # This looks potentially worth a revisit: should we have a system/bt dir?
The next build failed here:
[ 81% 3506/4283] target thumb C: libbt-vendor <= vendor/sprd/wcn/bt/libbt/src/bt_vendor_sprd.c
FAILED: /bin/bash -c "(PWD=/proc/self/cwd prebuilts/clang/host/linux-x86/clang-2690385/bin/clang -I device/samsung/sharkls-common/include -I vendor/sprd/wcn/bt/libbt/include -I system/bt/hci/include -I vendor/sprd/wcn/bt/libbt/conf/sprd/marlin/include -I vendor/sprd/wcn/bt/libbt -I /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates -I /mnt/Lavoro/halium/out/target/product/j3xlte/gen/SHARED_LIBRARIES/libbt-vendor_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem hardware/ril/include -isystem /mnt/Lavoro/halium/out/target/product/j3xlte/obj/include -isystem bionic/libc/arch-arm/include -isystem bionic/libc/include -isystem bionic/libc/kernel/uapi -isystem bionic/libc/kernel/common -isystem bionic/libc/kernel/uapi/asm-arm -isystem bionic/libm/include -isystem bionic/libm/include/arm -c -fno-exceptions -Wno-multichar -msoft-float -ffunction-sections -fdata-sections -funwind-tables -fstack-protector-strong -Wa,--noexecstack -Werror=format-security -D_FORTIFY_SOURCE=2 -fno-short-enums -no-canonical-prefixes -mcpu=cortex-a7 -mfpu=neon-vfpv4 -D__ARM_FEATURE_LPAE=1 -mfloat-abi=softfp -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -DNDEBUG -g -Wstrict-aliasing=2 -DNDEBUG -UDEBUG -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -nostdlibinc -target arm-linux-androideabi -target arm-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin -std=gnu99 -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing -fPIC -D_USING_LIBCXX -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type -MD -MF /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.d -o /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.o vendor/sprd/wcn/bt/libbt/src/bt_vendor_sprd.c ) && (cp /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.d /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.d >> /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.P; rm -f /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.d )"
In file included from vendor/sprd/wcn/bt/libbt/src/bt_vendor_sprd.c:31:
vendor/sprd/wcn/bt/libbt/include/bt_vendor_sprd.h:33:10: fatal error: 'bt_vendor_lib.h' file not found
#include "bt_vendor_lib.h"
^
1 error generated.
I had this file in external/bluetooth/bluedroid/hci/include/bt_vendor_lib.h, and I was about to add this directory to the vendor/sprd/wcn/bt/libbt/Android.mk file when I realized that this file was already including $(BDROID_DIR)/hci/include and defined BDROID_DIR := $(TOP_DIR)system/bt. So I understood that I should have changed my manifest file to check out the bluedroid repo under system/bt (which also meant that I was wrong in excluding the sprd-diff/system_bt.diff patch before). I also had a quick look at other manifest files, and I saw that they were typically getting system/bt from the LineageOS android_system_bt repo. Therefore I went to modify my manifest file, changing the bluedroid line to
<project path="system/bt" name="android_system_bt" remote="los" revision="cm-14.1" />
and removed the three vendor/nxp-nfc repositories I added a while before. Then I manually applied the patch in device/samsung/sharkls-common/patches/sprd-diff/system_bt.diff and I revisited the vendor/sprd/open-source/apps/engmode/bt/Android.mk to remove all the include lines I added before. The next build succeeded:
[...]
[ 99% 2250/2255] build /mnt/Lavoro/halium/out/target/product/j3xlte/obj/NOTICE.html
Combining NOTICE files into HTML
Combining NOTICE files into text
[ 99% 2254/2255] Target system fs image: /mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/systemimage_intermediates/system.img
BuildImage: in_dir = /mnt/Lavoro/halium/out/target/product/j3xlte/system, out_file = /mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/systemimage_intermediates/system.img
fs type is not ext4
Running: mkuserimg.sh -s /mnt/Lavoro/halium/out/target/product/j3xlte/system /mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/systemimage_intermediates/system.img ext4 system 2147483648 -D /mnt/Lavoro/halium/out/target/product/j3xlte/system -L system /mnt/Lavoro/halium/out/target/product/j3xlte/root/file_contexts.bin
make_ext4fs -s -T -1 -S /mnt/Lavoro/halium/out/target/product/j3xlte/root/file_contexts.bin -L system -l 2147483648 -a system /mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/systemimage_intermediates/system.img /mnt/Lavoro/halium/out/target/product/j3xlte/system /mnt/Lavoro/halium/out/target/product/j3xlte/system
Creating filesystem with parameters:
Size: 2147483648
Block size: 4096
Blocks per group: 32768
Inodes per group: 8192
Inode size: 256
Journal blocks: 8192
Label: system
Blocks: 524288
Block groups: 16
Reserved block group size: 127
Created filesystem with 1381/131072 inodes and 55467/524288 blocks
Running ['mkuserimg.sh', '-s', '/mnt/Lavoro/halium/out/target/product/j3xlte/system', '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/systemimage_intermediates/system.img', 'ext4', 'system', '2147483648', '-D', '/mnt/Lavoro/halium/out/target/product/j3xlte/system', '-L', 'system', '/mnt/Lavoro/halium/out/target/product/j3xlte/root/file_contexts.bin'] command, exit code = 0
[ 99% 2254/2255] Construct recovery from boot
failed to reconstruct target deflate chunk 1 [(null)]; treating as normal
chunk 0: type 0 start 0 len 5582858
chunk 1: type 2 start 5582858 len 8611584
chunk 2: type 0 start 9205682 len 673886
Construct patches for 3 chunks...
patch 0 is 212 bytes (of 5582858)
patch 1 is 4311951 bytes (of 3622824)
patch 2 is 207 bytes (of 673886)
chunk 0: normal ( 0, 5582858) 212
chunk 1: deflate ( 5582858, 8574717) 4311951 (null)
chunk 2: normal ( 14157575, 674057) 207
[100% 2255/2255] Install system fs image: /mnt/Lavoro/halium/out/target/product/j3xlte/system.img
/mnt/Lavoro/halium/out/target/product/j3xlte/system.img+/mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/recovery_patch_intermediates/recovery_from_boot.p maxsize=2192424960 blocksize=135168 total=197470210 reserve=22167552
make: Leaving directory '/mnt/Lavoro/halium'
#### make completed successfully (01:22 (mm:ss)) ####
Just to be on the safe side (to make sure I didn't have stale files from previous build attempts), I ran
mka clean mka halium-boot LANG=C mka systemimage
again, without encountering any errors. I also ran mka hybris-boot as per the Halium documentation, as it might be a good idea to test the Halium reference image first. hybris-boot.img was built successfully at the first try.
Flashing and testing the Halium reference image
Following the Halium documentation, I downloaded the Halium rootfs image:
mkdir rootfs cd rootfs wget 'http://bshah.in/halium/halium-rootfs-20170630-151006.tar.gz'
A small section of the documentation is dedicated on how to install the boot image for Samsung devices; it closely matches the steps I took before to install TWRP, which is comforting.
adb reboot download # much easier than the weird key combinations!
sudo heimdall flash --BOOT out/target/product/j3xlte/hybris-boot.img
The last command ended with this output:
Uploading BOOT 100% BOOT upload successful Ending session... ERROR: Failed to receive session end confirmation! Releasing device interface...
and the device was stuck on the same screen. I then thought of using TWRP to flash this device. I rebooted in download mode and installed TWRP as explained near the beginning of this blog post, then I uploaded my boot image to the SD card:
adb push out/target/product/j3xlte/hybris-boot.img /sdcard/
In TWRP, I did a wipe first: I wiped off the "System", "Data" and "cache" partitions (you can choose which partitions to wipe under the "Advanced wipe" option). Then I went back and pressed the "Install" button, then "Install Image" and picked my hybris-boot.img file (and chose "Boot" as target, of course). Make sure you don't reboot at this stage, or you'll lose adb access and the rest of the commands will fail.
I then installed the halium-install script:
git clone https://gitlab.com/JBBgameich/halium-install.git
I then downloaded the original Samsung ROM (see below for the link) and unzipped it in the rootfs directory I created before. That produced a file named J320FXXU0AQK2_J320FOXE0AQK2_J320FXXU0AQI1_HOME.tar.md5, from which I extracted the system.img and uploaded it to the device along with the halium rootfs, as explained in the "Halium guide":
tar xvf J320FXXU0AQK2_J320FOXE0AQK2_J320FXXU0AQI1_HOME.tar.md5 system.img ../halium-install/halium-install -p halium -v halium-rootfs-20170630-151006.tar.gz system.img
The last command complained about simg2img not being in $PATH, despite the fact that it was. A small change to the script fixed this. Once that was fixed it successfully pushed the files to the device, after asking me which password I wanted to set for the root user (on the phone): I chose a familiar "0000".
After rebooting, however, the device didn't go beyond the boot screen (with a red line warning about SELinux not being enabled). It just got stuck there, and was not even detected via USB.
Therefore, it's time to end this post with a "To be continued" message, and with a bonus section on how to restore the stock image below.
Restoring the Samsung stock ROM
The stock ROM can be downloaded from sammobile.com; you can restore the original image with this commands:
unzip J320*.zip tar xvf J320*.tar.md5 sudo heimdall flash --KERNEL boot.img --CACHE cache.img \ --HIDDEN hidden.img --l_fixnv2 nvitem.bin --PARAM param.lfs \ --pm_sys PM_sharkl_arm7.bin --RECOVERY recovery.img \ --SBOOT sboot.bin --SBOOT2 sboot2.bin --BOOT spl.img \ --l_modem SPRDCP.img --l_gdsp SPRDGDSP.img --l_ldsp SPRDLTEDSP.img \ --l_warm SPRDWDSP.img --SYSTEM system.img
If you were so crazy as to follow my steps, now running the command above is probably the wisest thing you could do. :-)
0 
0 
04 Jul 2019 3:47pm GMT
01 Jul 2019
Planet Maemo
Qbs and code coverage reports
You know that I'm not an early adopter. That's why it was only a couple of weeks ago when I decided to give Qbs a try, by using the good old Mappero (and its spin-off, Mappero Geotagger) as a test bench. Yes, I know that the Qt company is not going to maintain Qbs anymore in the future, but the little I knew about Qbs was enough to convince me that it's a project worth supporting. So, better late than never -- and hopefully the community (me included) will do a good job in keeping Qbs thriving.
Having Mappero build with Qbs was the simplest thing ever. The only issue I met was in building the unit tests, because I'm used to set the rpath on test executables in order to make it easy to run them uninstalled, and with qmake I achieved that with this:
QMAKE_RPATHDIR = $${QMAKE_LIBDIR}
In turns out that with Qbs you can do it in almost the same way, but for some reason I couldn't figure it out and I even reported a bug to which I got some nice suggestions, before eventually settling on this:
import qbs 1.0 Test { name: "path-test" files: [ "path-test.cpp", "path-test.h", "paths.qrc", ] Depends { name: "Mappero" } cpp.rpaths: cpp.libraryPaths // <-- this does the trick! }
It's surprisingly similar to how it's done in qmake, so it's not clear even to me why I didn't guess that immediately. Anyway, that was literally my only problem, and you can see the whole set of Qbs files I wrote by having a look at this commit.
Given how easy the migration was, I thought I should also try to add a code coverage report; that's not something I had in my qmake build either, but it's something I really want to have in all my newer projects.
Teaching Qbs to make a code coverage report
Unfortunately, my search for examples on how to have Qbs prepare a coverage report was mostly insuccessful, but thanks to some amazing help from Christian in the #qbs IRC channel, this was not hard to achieve. So, I hope to be of some help myself too, by sharing how this works.
First of all, it must be said that Qbs doesn't know anything about code coverage, at all. However, it's possible (and often easy) to extend Qbs by adding your own Product with its own set of build rules, so here's the CoverageReport item for Mappero (though, it should be general enough to be reusable in your own project):
import qbs Product { name: "coverage" property string outputDirectory: "coverage-html" property stringList extractPatterns: [] builtByDefault: false files: ["**"] type: ["coverage.html"] Depends { productTypes: ["autotest-result"] } Rule { multiplex: true explicitlyDependsOnFromDependencies: ["autotest-result"] outputFileTags: "coverage.html" requiresInputs: false prepare: { var commands = [] var captureCmd = new Command("lcov", [ "--directory", project.sourceDirectory, "--capture", "--output-file", "coverage.info", "--no-checksum", "--compat-libtool", ]); captureCmd.description = "Collecting coverage data"; captureCmd.highlight = "coverage"; captureCmd.silent = false; commands.push(captureCmd); var extractArgs = [] for (var i = 0; i < product.extractPatterns.length; i++) { extractArgs.push("--extract"); extractArgs.push("coverage.info"); extractArgs.push(product.extractPatterns[i]); } if (product.extractPatterns.length > 0) { extractArgs.push("-o"); extractArgs.push("coverage.info"); var extractCmd = new Command("lcov", extractArgs); extractCmd.description = "Extracting coverage data"; extractCmd.highlight = "coverage"; extractCmd.silent = false; commands.push(extractCmd); } var filterCmd = new Command("lcov", [ "--remove", "coverage.info", 'moc_*.cpp', "--remove", "coverage.info", 'qrc_*.cpp', "--remove", "coverage.info", '*/tests/*', "-o", "coverage.info", ]); filterCmd.description = "Filtering coverage data"; filterCmd.highlight = "coverage"; filterCmd.silent = false; commands.push(filterCmd); var genhtmlCmd = new Command("genhtml", [ "--prefix", project.sourceDirectory, "--output-directory", product.outputDirectory, "--title", "Code coverage", "--legend", "--show-details", "coverage.info", ]); genhtmlCmd.description = "Generate HTML coverage report"; genhtmlCmd.highlight = "coverage"; genhtmlCmd.silent = false; commands.push(genhtmlCmd); return commands; } } }
The most important thing here are the references to the autotest-result tag: this is the tag used by the AutotestRunner Qbs item, which is responsible for running the unit tests. Referencing its product's tag in the Depends item and in the explicitlyDependsOnFromDependencies properties ensures that "building" our product will cause the unit tests to run. Other needed bits are the requiresInputs: false property, which means that our rule doesn't have any required inputs, and the builtByDefault: false property, which says that our coverage report should not be generated when just typing qbs. Instead, to run the tests and get the code coverage report one will have to request it explicitly, by typing
qbs -p coverage
The prepare property of the Rule is where the commands to generate the code coverage report are defined. Here we can use the Command item to invoke external programs, and we return a list of such items, so that the commands will be executed in sequence. Note that here I'm using lcov and expecting to find the coverage data produced by gcov, so this is probably not portable outside of Linux/gcc.
Using the CoverageReport item is quite easy: you just need to declare it, and specify which paths contain the coverage data that you are interested in (otherwise, lcov will collect data from all object files that it find under the build directory, which might not be what you desire):
CoverageReport {
condition: project.enableCoverage
extractPatterns: [ '*/src/*.cpp', '*/lib/*.cpp' ]
}
There's little more than that to be done. Of course, you need to find a way to pass the --coverage option to gcc when building your products, and for this I created a small buildconfig module in qbs/modules/buildconfig/BuildConfig.qbs which I depend on in all products which I wish to build with coverage enabled:
import qbs Module { cpp.cxxFlags: project.enableCoverage ? ["--coverage"] : undefined cpp.dynamicLibraries: project.enableCoverage ? ["gcov"] : undefined Depends { name: "cpp" } }
If all this looks scary, you should probably have a look at the diff which shows how I added code coverage reporting to qbs: hopefully you'll find that it's not that complex, after all.
I hope that Qbs users will find this interesting, and possibly improving my setup. Ideally we should try to get something like this part of Qbs itself, but portability outside of Linux / gcc is going to be an issue.
1 0
01 Jul 2019 1:42pm GMT
13 Jun 2019
Planet Maemo
WIP: changing the backend for contacts in Ubports
More than one year has passed since the initial announcement of my plan to investigate using a different backend for contact storage. If you want to get a better understanding of the plan, that mail is still a good read -- not much has changed since them, planning wise.
The reason for this blog post is to give a small update on what has happened since then, and as a start nothing can be better than a couple of screenshots:
Adding CardDAV accounts in the Addressbook application 
Aggregated contact details from multiple sourcesIn other words, that means that contact synchonisation works, both with the new CardDAV protocol (for which we'll have preconfigured setups for NextCloud and OwnCloud accounts) and with Google Contacts, for which we are now using a different engine. What you see in the second screenshot (although indeed it's not obvious at all) is that the new qtcontacts-sqlite backend performs automatic contact merging based on some simple heuristics, meaning that when synchonising the same contact from multiple sources you should not happen to find a multitude of semi-identical copies of the contact, but a single one having all the aggregated details.
Before you get too excited, I have to say that this code is pre-alpha quality and that it's not even available for testing yet. The next step is indeed to setup CI so that the packaggqes get automatically built and published to a public repository, at which point I'll probably issue another update here in my blog.
The boring stuff
And now some detail for those who might wonder why this feature is not ready yet, or would like to get an idea on the time-frame for its completion.
Apart from a chronical lack of time from my part, the feature complexity is due to the large number of components involved:
- qtcontacts-sqlite: the QtContacts backend we are migrating to. This is a backend for the QtContacts API (used by our Addressbook application) which uses a SQLite database as storage for your contacts.
- buteo-sync-plugin-carddav: the CardDAV plugin for Buteo (our synchronisation manager). This plugin is loaded by Buteo and synchronises the contacts between a CardDAV remote source and the
qtcontacts-sqlitedatabase. - buteo-sync-plugins-social: a Buteo plugin which can synchronise contacts from a multitude of sources, including Google, Facebook and Vk. At the moment we only care about Google, but once this feature has landed we can easily extend it to work with the other two as well.
- address-book-app: this is our well-known Contacts application. It needs some minor changes to adapt to the
qtcontacts-sqlitebackend and to support the creation of new CardDAV, NextCloud and OwnCloud accounts. - QtPim: the contacts and calendar API developed by the Qt project. Our Contacts application is using the front-end side of this API, and the
qtcontacts-sqlitecomponent implements the backend side. There are some improvements proposed by Jolla, which we need to include in order to support grouping contacts by their initials.
The other tricky aspect is that the first three projects are maintained by Jolla as part of the Sailfish OS, and while on one side this means that we can share the development and maintenance burden with Jolla, on the other side of the coin it means that we need to apply extra care when submitting changes, in order not to step on each other's shoes. Specifically, Sailfish OS is using a much older version of QtPim than Ubports is, and the APIs between the two versions have changes in an incompatible version, so that it's nearly impossible to have a single code base working with both versions of QtPim. Luckily git supports branches, and Chris from Jolla was kind enough to create a branch for us in their upstream repository where I've proposed our changes (and they are a lot!).
However, this is not as bad as it sounds, and the fact that I have a roughly working version on my development device is a good sign that things are moving forwards.
0 0
13 Jun 2019 2:59pm GMT
12 Nov 2011
Planet Linux-to-go
Paul 'pfalcon' Sokolovsky: Shopping for 3D TV...
Shopping for 3D TV (again), few findings:
- There are no non-LED 3D TVs for sale in this part of world anymore (in particular, my older favorite, LE40C750 is no longer available)
- Samsung D6xxx series (the cheapest) were caught red-handed not providing FullHD resolution in 3D: via Samygo. (Note from myself: Samsung was caught because lots of people actually buy their stuff, what one can think about other vendors?)
- LG goes out of line of vendors with 3D shutter glasses technology with their FPR ("Cinema 3D" in marketing speak) technology. It uses passive (no power needed) polarized glasses, like RealD cimenas. Announced April (on these longitudes), already in local shops at not-so-indecent prices. FPR is Film Patterned Retarder, and I hope the last word is spelled right, because I have high hopes for this tech. It shares the same issue as Samsung D6xxx: there won't FullHD here for sure, but at least it's official and there're other benefits.
12 Nov 2011 6:55pm GMT
Paul 'pfalcon' Sokolovsky: Hacking Luxeon SP-1
I finally going to get Arduino, and while I'm choosing flavor and waiting for it, I can't help but disassembling all devices I have at home, each time speaking: "This must have Arduino inside!" (meaning of course that I expect it to be based on general-purpose MCU). Gosh, I usually get "blob chip" (uncased chip with blob of epoxy on top).
Well, I finally had my expectations fulfilled - Luxeon SP-1 voltage stabilizer/cutter features ATMEGA48V-10PU (Flash: 4k, EEPROM: 256, RAM:512). Not only that, it is installed in DIP socket! Buy from Luxeon, they're hacker-friendly ;-).
I bought the device actually for a wattmeter it features (which fact is hard to figure out from common specs found in the shops, I accidentally read somebody mentioning it on a forum). The wattmeter is of course not bright - for a lamp rated 100W it shows 88W, and for more powerful equipment (like perforator) understates wattage even more (maybe it's difference between real and apparent power factor).
Still, for $17 you get Arudino-alike with voltage/current sensor and hacking possibility. Woot!
BOM:
High-power board:
- Relay: Coil: 24VDC, 5A/240VAC
- 7805
- Coil transformer LR-019B
MCU board:
- ATMEGA48V-10PU
- 2 buttons
- 2 LEDs (red & green)
- 3-digit 7-segment LED indicator
12 Nov 2011 5:58pm GMT
10 Nov 2011
Planet Linux-to-go
Paul 'pfalcon' Sokolovsky: Links for November 2011
Kindle:
- Amazon Kindle official GPL source code
- Kindle keyboard shortcuts and other reference
- Latest jailbreak appears to be one from NiLuJe (forum). It's known to work with firmware 3.3. Hacks/updates should be copied to Kindle USB drive root.
- SirAlex's page
- Intro for writing kindlets
Linux kernel module tricks:
- How to bind Linux driver to device whose IDs it doesn't know (yet), without recompiling
- Building a kernel module separately
- (build module from a kernel source tarball using currently installed kernel config)
10 Nov 2011 3:21pm GMT
19 Oct 2011
Planet OpenEZX
Antonio Ospite: Gnome 3: go to Shell? Not just yet, thanks.
In Debian Unstable the transition to Gnome 3 is taking place; when Gnome 3.0 firstly came out some unnamed geeky users complained loudly about the design decisions of the development team to push strongly towards gnome-shell as a new default UI; gnome-shell was designed focusing on usability (usability is a metric relative to a certain target audience BTW) and simplicity, hiding a lot of details from the users. Obviously you can never make everyone happy so some of us simply happened to be "out of target": you know us computer people (*cough cough*), we like to be in charge and control The Machine... I must admit I still don't have a definitive opinion about the gnome-shell concept, for now I just know that it does not suit me; I am going to try it eventually, maybe I'll get used to it, but in the mean time I need my desktop back just like I shaped it through the years; can this be done without loosing all the good Gnome technologies (Empathy over all of them)?
To be completely fair I have to say that there is little to complain about with Gnome developers, we can still get our good old GNOME desktop fully back by using the fall-back mode based on gnome-panel and live happily ever after, let's take a look at how this can be accomplished.
NOTE: GNOME people state that the fall-back mode is meant for systems with older graphic cards which cannot run gnome-shell, however it can very well be seen as a good opportunity for those who do not want to run gnome-shell just yet.
Getting back to the topic: some minor touches are needed to make the panel look more like what we are used to, maybe some of these settings could even become default for fall-back mode, we'll see.
First, enable fall-back mode (on Debian there is a dedicated session you can choose from the Log-in Manager for that) and change some desktop settings, in a terminal type:
$ gsettings set org.gnome.desktop.session session-name 'gnome-fallback' $ gsettings set org.gnome.desktop.interface 'menus-have-icons' true $ gsettings set org.gnome.desktop.interface 'buttons-have-icons' true $ gsettings set org.gnome.desktop.background 'show-desktop-icons' true
gnome-tweak-tool can be used for some of these settings like shown in the attached images.
Then rearrange the applets on the panel as you please (use Alt-RightClick to access the panel properties), and fix the theming using this patch to have a light panel again (against gnome-themes-standard=3.0.2-1):
$ mkdir $HOME/.themes $ cd $HOME/.themes $ cp -r /usr/share/themes/Adwaita Adwaita-fallback $ cd Adwaita-fallback $ patch -p1 < $HOME/adwaita-fallback-panel-theme.patch $ gsettings set org.gnome.desktop.interface 'gtk-theme' 'Adwaita-fallback'
Some final touches for the Metacity window manager and to the clock applet, and we are all set:
$ gconftool-2 --type string --set /apps/metacity/general/focus_mode mouse $ gconftool-2 --type boolean --set /apps/metacity/general/compositing_manager true $ gconftool-2 --type string --set /apps/panel3-applets/clock/custom_format '<span color="#333">%a %d %b</span> <b>%H:%M</b>' $ gconftool-2 --type string --set /apps/panel3-applets/clock/format custom
Ah, in the new gnome-panel based on Gtk3 there are still some details to take care of, I hope issues like that will be addressed and that the panel will be supported for quite some time.
19 Oct 2011 9:37pm GMT
09 Jun 2011
Planet OpenEZX
Michael Lauer: The Eagle Has Landed!
After letting us wait for a bit longer than scheduled (13 days), the hospital initiated the contractions. For the first couple of hours, everything went just perfect, but then the little one got stuck on the way and we had to resort to a cesarean section. Lara Marie Lauer was born 8th of June at 04:41 (AM) with 3460 gramms and 49 cm.
Mummy was still on intensive care and so they gave her to me. I can't express the feelings I had in this very moment. I'm still kind of overwhelmed every time I see her. Thanks for all of you who waited anxiously with me and those who prayed for us. The most important tasks for the near future is getting Mummy to recover and Lara Marie to become accustomed to us and the rest of the outside world.
Please bear with me if in the next time I'm not as responsive as usually 
09 Jun 2011 4:06pm GMT
30 May 2011
Planet OpenEZX
Michael Lauer: German Post on time!
And now for something completely different… while we are all waiting for my baby to arrive (who was scheduled for 25th of May), she just received her first greeting card - together with a personalized bib and a towel (with integrated hood - pretty fancy!) from my good friends at #openmoko-cdevel.
Guys, seeing this card was very heartwarming - it means a lot to me that you share my anticipation, thanks a lot! And I'm 100% sure she will appreciate her gifts… now let's cross fingers it won't take much longer… waiting is the hardest part of it
Yours,
Mickey.
30 May 2011 8:54am GMT