24 Nov 2020

feedTalkAndroid

Hallmark Movies Now comes to YouTube TV on a 7-day trial, $5.99 monthly thereafter

It's getting close to that time of year and with 2020 being what it is, any sort of feel-good experience is worth trying out. This means that the launch of Hallmark Movies Now on YouTube TV has come at a great time with the added bonus of being able to try it out on a […]


Come comment on this article: Hallmark Movies Now comes to YouTube TV on a 7-day trial, $5.99 monthly thereafter

Visit TalkAndroid

24 Nov 2020 10:40pm GMT

This is when your Huawei smartphone or tablet will get the EMUI 11 update

Huawei's Mate 40 Pro launched in September running EMUI 11 and now the handset maker has posted a timeline that reveals when users can expect their handsets and tablets to receive the update. The list covers unlocked devices ranging from the Mate 20 Series to the MatePad Pro tablet in just about every region. The […]


Come comment on this article: This is when your Huawei smartphone or tablet will get the EMUI 11 update

Visit TalkAndroid

24 Nov 2020 10:13pm GMT

[Deal] Save 67% on the FIXD Car Diagnostic Sensor through until Nov. 30

Taking your car to a dealer or the local mechanic can be a stressful affair, especially if you aren't actually sure what is wrong with the car. This is where the FIXD car repair sensor comes into its own, alerting you to perform scheduled maintenance, find out if the issue is severe enough to be […]


Come comment on this article: [Deal] Save 67% on the FIXD Car Diagnostic Sensor through until Nov. 30

Visit TalkAndroid

24 Nov 2020 9:29pm GMT

23 Nov 2020

feedAndroid Developers Blog

MAD Skills — Become an Android App Bundle expert

Posted by Ben Weiss, Developer Relations Engineer

Android image

The Android App Bundle series of Modern Android Development has just concluded. We finished off with a live Q&A session. I was joined by Chet Haase, Wojtek Kaliciński, and Iurii Makhno to go over bundles of questions from the #AskAndroid hashtag on Twitter as well as from the chat during the live stream.

But let's rewind for a moment and take it from the top.

Introduction to Android App Bundles

In the inaugural episode Wojtek sets the tone for this series by talking about why app bundles matter to you and your app.

Everything to know about Play App Signing

In this episode you learn how to opt into Play App Signing by joining Wojtek on a journey through the Play Console. After watching this video you will have learned what options are available to you when opting into Play App Signing.

Alongside this video we recommend taking a look at the FAQ about Play App Signing, the app signing Android docs and the Play Console's Play App Signing help page.

Building your first app bundle

Now it's time to learn how to build and upload your first Android App Bundle.

In this episode, I take you through the process of building a bundle using Android Studio and the command line interface.

Instead, you can read the write up of this episode.

Accompanying this episode, take a look at the app bundles documentation.

Configuring your app for Play Feature Delivery

Here you'll learn about delivery options. From install time, to conditional delivery to on demand delivery. I cover it all. And we'll take an excursion through the sample on GitHub.

This episode also is available as an article for you to read. Additionally, the PlayCore guide is a valuable resource.

Testing app bundles with bundletool and the Play Console

Have you wondered about how you can test your app bundle? No more wondering. Wojtek takes you through testing your app bundle locally and with the Play Console.

You can read up on this episode's content in the accompanying article and the guide to testing your Android App Bundle.

Additionally we have guidance on developer tools on the Play Console and the Play Console help page for internal app sharing available for you.

And, if you want to download bundletool, here is where to find it.

Big savings with Android App Bundles

Android GDE Angélica Oliveira tells us about the process and the impressive size savings her company saw when switching over to Android App Bundles.

Live Q&A session

We asked for your questions on Twitter. You replied, using the #AskAndroid hashtag.

And you continued asking questions during the live Q&A session.

Chet then got Wojtek, Iurii and myself in front of the camera for a live Q&A to answer your questions.

Reminder: App bundles will be mandatory for new apps from August 2021

Read more about the 2021 API level bump and app bundle requirement

23 Nov 2020 10:29pm GMT

The future of Kotlin Android Extensions

Posted by David Winer, Product Manager

The Android Kotlin Extensions Gradle plugin (not to be confused with Android KTX) was released in 2017 and brought two new conveniences to Android development in Kotlin:

Since then, we have released View Binding for Android, an officially supported library that has deep integration with the Android build toolchain and provides similar functionality as Kotlin synthetics. While we continue to recommend Parcelize, a number of drawbacks have appeared with using Kotlin synthetics:

JetBrains originally developed the Android Kotlin Extensions plugin, and together we have discussed the pros and cons of continuing to maintain synthetics: we strive to ensure long term support for APIs where we can but want to guide developers towards best practices that make for healthy codebases and, ultimately, happy users.

Over the course of the next year, our teams will be jointly deprecating synthetics in favor of continuing to support our recommended option, View Binding. Here's what that means:

The deprecation period starts with Kotlin 1.4.20, released today. android-kotlin-extensions will be removed in a future Kotlin release during or after September 2021. Long term, we will continue to maintain the kotlin-parcelize plugin, and you can continue to file issues on Parcelize in the Android Studio issue tracker.

23 Nov 2020 5:02pm GMT

19 Nov 2020

feedAndroid Developers Blog

New Android App Bundle and target API level requirements in 2021

Posted by Hoi Lam, Developer Relations Engineer, Android Platform

Android app bundle image

In 2021, we are continuing with our annual target API level update, requiring new apps to target API level 30 (Android 11) in August and in November for all app updates. In addition, as announced earlier this year, Google Play will require new apps to use the Android App Bundle publishing format. This brings the benefits of smaller apps and simpler releases to more users and developers and supports ongoing investment in advanced distribution.

Over 750,000 apps and games already publish to production on Google Play using app bundles. Top apps switching save an average size of 15% versus a universal APK. Users benefit from smaller downloads and developers like Netflix and Riafy see higher install success rates, which is especially impactful in regions with more entry level devices and slower data speeds. Developers switching can use advanced distribution features such as Play Asset Delivery and Play Feature Delivery. We value your feedback and plan to introduce further features and options for Play App Signing and Android App Bundles before the switchover.


Requirements for new apps

From August 2021, the Google Play Console will require all new apps to:


Requirements for updates to existing apps

From November 2021, updates to existing apps will be required to target API level 30 or above and adjust for behavioral changes in Android 11. Existing apps that are not receiving updates are unaffected and can continue to be downloaded from the Play Store.

Requirements for instant experiences

The switch to Android App Bundle delivery will also impact instant experiences using the legacy Instant app ZIP format. From August 2021, new instant experiences and updates to existing instant experiences will be required to publish instant-enabled app bundles.


Moving forward together

Here is a summary of all the changes:


TYPE OF RELEASE

REPLACED

REQUIRED AUG 2021

New apps
on Google Play

APK

Android App Bundle (AAB)

Target API level set to 29+

Target API level set to 30+

Expansion files (OBBs)

Play Asset Delivery or
Play Feature Delivery

TYPE OF RELEASE

REPLACED

REQUIRED NOV 2021

Updates to existing apps
on Google Play

No new publishing format requirement

Target API level set to 29+

Target API level set to 30+



Wear OS apps are not subject to the new target API level requirement.

Apps can still use any minSdkVersion, so there is no change to your ability to build apps for older Android versions.

To learn more about transitioning to app bundles, watch our new video series: modern Android development (MAD) skills. We are extremely grateful for all the developers who have adopted app bundles and API level 30 already. We look forward to advancing the Android platform together with you.

19 Nov 2020 6:09pm GMT

18 Nov 2020

feedPlanet Maemo

Self-built NAS for Nextcloud hosting

With Google cutting its unlimited storage and ending the Play Music service, I decided to use my own Nextcloud more seriously.
In part because Google forced all its competitors out of the market, but mostly because I want to be independent of any cloudy services.

The main drawback of my existing Nextcloud setup, that I have written about here, was missing redundancy; the nice thing about putting your stuff in the cloud is that you do not notice if one of the storage devices fails - Google will take care of providing you with a backup copy of your data.

Unfortunately, the Intel NUC based build I used, while offering great power efficiency did not support adding a second HDD to create a fail-safe RAID1 setup. Therefore I had to upgrade.

As I still wanted to keep things power-efficient in a small form-factor, my choice fell on the ASRock DeskMini series. Here, I went with the AMD Variant (A300) in order to avoid paying the toll of spectre mitigations with Intel (resulting in just 80% of baseline performance).

Size comparison between the A300 and a Gigabyte BRIX

The photos above show you the size difference, which is considerable - yet necessary to cram two 2.5″ SATA drives next to each other.
Here, keep in mind that while the NUC devices have their CPU soldered on, we are getting the standardized STX form-factor with the A300, which means you can replace and upgrade the mainboard and the CPU as you wish, while with a NUC you are basically stuck with what you bought initially.

The full config of the build is as follows and totals at about 270€

Note, that I deliberately chose SSDs from different vendors to reduce the risk of simultaneous failure.
Also, while the 3000G is not the fastest AMD CPU, it is sufficient to host nextcloud and is still a nice upgrade from the Intel Celeron I used previously.
Furthermore, its 35W TDP nicely fits the constrained cooling options. Note, that you can limit for Ryzen 3/5 CPUs to 35W in the BIOS as well, so there is not need to get their GE variants.
However, for a private server you probably do not need that CPU power anyway, so just go with the Athlon 3000G for half the price.

Unfortunately, the A300 system is not designed for passive cooling and comes with a quite annoying CPU fan. To me the fan coming with the Athlon 3000G was less annoying, so I used that instead.
Anyway, you should set the fan RPM to 0% below 50° C in the BIOS, which results in 800 RPM and is unhearable while keeping the CPU reasonably cool.

Power Consumption

As the machine will run 24/7, power consumption is an important factor. As a rule of thumb you can calculate with 1€ for 1 Watt drained non-stop for 1 Year.

The 35W TDP gives us a upper limit of what the system will consume on persistent load - however the more interesting measure is the idle consuption as thats the state the system will be most of the time.

As I already tried some builds with different architectures, we have some interesting values to compare to, putting the A300 build in perspective

Build CPU Idle Load
Odroid U3 Exynos4412 3.7 W 9 W
Gigabyte BRIX Intel N3350 4.5 W 9.6 W
A300 Athlon 3000G 7.3 W 33.6 W

While you can obviously push the system towards 35W by with multiple simultaneous users, the 7.3 W idle consumption is quite nice.
Keep in mind, that the A300 was measured with two SATA drives operating as RAID1. If you only use one you can subtract 1W - at which point it is only 1.5 W away from the considerably weaker NUC system.

You might now wonder, whether the load or the idle measure is closer to the typical consumption. For this I measured the consumption for one day, which totalled at 0.18 kWh - or 7.5 Watts.

Power optimizations

To reach that 7.3 W idle, you need to tune some settings though. The most important one and luckily the easiest to fix is using a recent kernel.
If you are on Ubuntu 18.04, update to 20.04 or install the hwe kernel (5.4.0) - it saves you 4 Watts (11.3 to 7.3).

For saving about 0.5 watts, you can downgrade the network interface from 1Gbit to 100Mbit by executing

ethtool -s enp2s0 speed 100 duplex full autoneg on

Additionally, you can use Intels powertop to tune your system settings for power saving as

powertop --auto-tune

0 Add to favourites0 Bury

18 Nov 2020 8:16pm GMT

12 Nov 2020

feedPlanet Maemo

Imaginario 0.10

Today I released Imaginario 0.10. No bigger changes there, but two important bugfixes.

0 Add to favourites0 Bury

12 Nov 2020 4:12pm GMT

16 Oct 2020

feedPlanet Maemo

Figuring out corrupt stacktraces on ARM

If you're developing C/C++ on embedded devices, you might already have stumbled upon a corrupt stacktrace like this when trying to debug with gdb:

(gdb) bt 
#0  0xb38e32c4 in pthread_getname_np () from /home/enrique/buildroot/output5/staging/lib/libpthread.so.0
#1  0xb38e103c in __lll_timedlock_wait () from /home/enrique/buildroot/output5/staging/lib/libpthread.so.0 
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

In these cases I usually give up gdb and try to solve my problems by adding printf()s and resorting to other tools. However, there are times when you really really need to know what is in that cursed stack.

ARM devices subroutine calls work by setting the return address in the Link Register (LR), so the subroutine knows where to point the Program Counter (PC) register to. While not jumping into subroutines, the values of the LR register is saved in the stack (to be restored later, right before the current subroutine returns to the caller) and the register can be used for other tasks (LR is a "scratch register"). This means that the functions in the backtrace are actually there, in the stack, in the form of older saved LRs, waiting for us to get them.

So, the first step would be to dump the memory contents of the backtrace, starting from the address pointed by the Stack Pointer (SP). Let's print the first 256 32-bit words and save them as a file from gdb:

(gdb) set logging overwrite on
(gdb) set logging file /tmp/bt.txt
(gdb) set logging on
Copying output to /tmp/bt.txt.
(gdb) x/256wa $sp
0xbe9772b0:     0x821e  0xb38e103d   0x1aef48   0xb1973df0
0xbe9772c0:      0x73d  0xb38dc51f        0x0          0x1
0xbe9772d0:   0x191d58    0x191da4   0x19f200   0xb31ae5ed
...
0xbe977560: 0xb28c6000  0xbe9776b4        0x5      0x10871 <main(int, char**)>
0xbe977570: 0xb6f93000  0xaaaaaaab 0xaf85fd4a   0xa36dbc17
0xbe977580:      0x130         0x0    0x109b9 <__libc_csu_init> 0x0
...
0xbe977690:        0x0         0x0    0x108cd <_start>  0x0
0xbe9776a0:        0x0     0x108ed <_start+32>  0x10a19 <__libc_csu_fini> 0xb6f76969  
(gdb) set logging off
Done logging to /tmp/bt.txt.

Gdb already can name some of the functions (like main()), but not all of them. At least not the ones more interesting for our purpose. We'll have to look for them by hand.

We first get the memory page mapping from the process (WebKit's WebProcess in my case) looking in /proc/pid/maps. I'm retrieving it from the device (named metro) via ssh and saving it to a local file. I'm only interested in the code pages, those with executable ('x') permissions:

$ ssh metro 'cat /proc/$(ps axu | grep WebProcess | grep -v grep | { read _ P _ ; echo $P ; })/maps | grep " r.x. "' > /tmp/maps.txt

The file looks like this:

00010000-00011000 r-xp 00000000 103:04 2617      /usr/bin/WPEWebProcess
...
b54f2000-b6e1e000 r-xp 00000000 103:04 1963      /usr/lib/libWPEWebKit-0.1.so.2.2.1 
b6f6b000-b6f82000 r-xp 00000000 00:02 816        /lib/ld-2.24.so 
be957000-be978000 rwxp 00000000 00:00 0          [stack] 
be979000-be97a000 r-xp 00000000 00:00 0          [sigpage] 
be97b000-be97c000 r-xp 00000000 00:00 0          [vdso] 
ffff0000-ffff1000 r-xp 00000000 00:00 0          [vectors]

Now we process the backtrace to remove address markers and have one word per line:

$ cat /tmp/bt.txt | sed -e 's/^[^:]*://' -e 's/[<][^>]*[>]//g' | while read A B C D; do echo $A; echo $B; echo $C; echo $D; done | sed 's/^0x//' | while read P; do printf '%08x\n' "$((16#"$P"))"; done | sponge /tmp/bt.txt

Then merge and sort both files, so the addresses in the stack appear below their corresponding mappings:

$ cat /tmp/maps.txt /tmp/bt.txt | sort > /tmp/merged.txt

Now we process the resulting file to get each address in the stack with its corresponding mapping:

$ cat /tmp/merged.txt | while read LINE; do if [[ $LINE =~ - ]]; then MAPPING="$LINE"; else echo $LINE '-->' $MAPPING; fi; done | grep '/' | sed -E -e 's/([0-9a-f][0-9a-f]*)-([0-9a-f][0-9a-f]*)/\1 - \2/' > /tmp/mapped.txt

Like this (address in the stack, page start (or base), page end, page permissions, executable file load offset (base offset), etc.):

0001034c --> 00010000 - 00011000 r-xp 00000000 103:04 2617 /usr/bin/WPEWebProcess
...
b550bfa4 --> b54f2000 - b6e1e000 r-xp 00000000 103:04 1963 /usr/lib/libWPEWebKit-0.1.so.2.2.1 
b5937445 --> b54f2000 - b6e1e000 r-xp 00000000 103:04 1963 /usr/lib/libWPEWebKit-0.1.so.2.2.1 
b5fb0319 --> b54f2000 - b6e1e000 r-xp 00000000 103:04 1963 /usr/lib/libWPEWebKit-0.1.so.2.2.1
...

The addr2line tool can give us the exact function an address belongs to, or even the function and source code line if the code has been built with symbols. But the addresses addr2line understands are internal offsets, not absolute memory addresses. We can convert the addresses in the stack to offsets with this expression:

offset = address - page start + base offset

I'm using buildroot as my cross-build environment, so I need to pick the library files from the staging directory because those are the unstripped versions. The addr2line tool is the one from the buldroot cross compiling toolchain. Written as a script:

$ cat /tmp/mapped.txt | while read ADDR _ BASE _ END _ BASEOFFSET _ _ FILE; do OFFSET=$(printf "%08x\n" $((0x$ADDR - 0x$BASE + 0x$BASEOFFSET))); FILE=~/buildroot/output/staging/$FILE; if [[ -f $FILE ]]; then LINE=$(~/buildroot/output/host/usr/bin/arm-buildroot-linux-gnueabihf-addr2line -p -f -C -e $FILE $OFFSET); echo "$ADDR $LINE"; fi; done > /tmp/addr2line.txt

Finally, we filter out the useless [??] entries:

$ cat /tmp/bt.txt | while read DATA; do cat /tmp/addr2line.txt | grep "$DATA"; done | grep -v '[?][?]' > /tmp/fullbt.txt

What remains is something very similar to what the real backtrace should have been if everything had originally worked as it should in gdb:

b31ae5ed gst_pad_send_event_unchecked en /home/enrique/buildroot/output5/build/gstreamer1-1.10.4/gst/gstpad.c:5571 
b31a46c1 gst_debug_log en /home/enrique/buildroot/output5/build/gstreamer1-1.10.4/gst/gstinfo.c:444 
b31b7ead gst_pad_send_event en /home/enrique/buildroot/output5/build/gstreamer1-1.10.4/gst/gstpad.c:5775 
b666250d WebCore::AppendPipeline::injectProtectionEventIfPending() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.cpp:1360 
b657b411 WTF::GRefPtr<_GstEvent>::~GRefPtr() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/DerivedSources/ForwardingHeaders/wtf/glib/GRefPtr.h:76 
b5fb0319 WebCore::HTMLMediaElement::pendingActionTimerFired() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WebCore/html/HTMLMediaElement.cpp:1179 
b61a524d WebCore::ThreadTimers::sharedTimerFiredInternal() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WebCore/platform/ThreadTimers.cpp:120 
b61a5291 WTF::Function<void ()>::CallableWrapper<WebCore::ThreadTimers::setSharedTimer(WebCore::SharedTimer*)::{lambda()#1}>::call() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/DerivedSources/ForwardingHeaders/wtf/Function.h:101 
b6c809a3 operator() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:171 
b6c80991 WTF::RunLoop::TimerBase::TimerBase(WTF::RunLoop&)::{lambda(void*)#1}::_FUN(void*) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:164 
b6c80991 WTF::RunLoop::TimerBase::TimerBase(WTF::RunLoop&)::{lambda(void*)#1}::_FUN(void*) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:164 
b2ad4223 g_main_context_dispatch en :? 
b6c80601 WTF::{lambda(_GSource*, int (*)(void*), void*)#1}::_FUN(_GSource*, int (*)(void*), void*) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:40 
b6c80991 WTF::RunLoop::TimerBase::TimerBase(WTF::RunLoop&)::{lambda(void*)#1}::_FUN(void*) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:164 
b6c80991 WTF::RunLoop::TimerBase::TimerBase(WTF::RunLoop&)::{lambda(void*)#1}::_FUN(void*) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:164 
b2adfc49 g_poll en :? 
b2ad44b7 g_main_context_iterate.isra.29 en :? 
b2ad477d g_main_loop_run en :? 
b6c80de3 WTF::RunLoop::run() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:97 
b6c654ed WTF::RunLoop::dispatch(WTF::Function<void ()>&&) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/RunLoop.cpp:128 
b5937445 int WebKit::ChildProcessMain<WebKit::WebProcess, WebKit::WebProcessMain>(int, char**) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WebKit/Shared/unix/ChildProcessMain.h:64 
b27b2978 __bss_start en :?

I hope you find this trick useful and the scripts handy in case you ever to resort to examining the raw stack to get a meaningful backtrace.

Happy debugging!

0 Add to favourites0 Bury

16 Oct 2020 7:07pm GMT

25 Sep 2017

feedPlanet Openmoko

Holger "zecke" Freyther: Brain dump what fascinates me

A small brain dump of topics that currently fascinate me. These are mostly pointers and maybe it is interesting to follow it.

Books/Reading:

My kobo ebook reader has the Site Reliability Engineering book and I am now mostly done. It is kind of a revelation and explains my interest to write code but also to operate infrastructure (like struggling with ruby, rmagick, nginx…). I am interested in backends since… well ever. The first time I noticed it when we talked about Kolab at LinuxTag and I was more interested in the backend than the KDE client. At sysmocom we built an IoT product and the backend was quite some fun, especially the scale of one instance and many devices/users, capacity planning and disk commissioning, lossless upgrades.

It can be seen in my non FOSS SS7 map work on traffic masquerading and transparent rewriting. It is also clear to see which part of engineering is needed for scale (instead of just installing and restarting servers).

Lang VM design

One technology that made Java fast (Hotspot) and has seen its way into JavaScript is dynamic optimization. Most Just in Time Compilers start with generating native code per method, either directly or after the first couple of calls when the methods size is significant enough. The VM records which call paths are hot, which types are used and then can generate optimized code (e.g. specialized for integers, remove type checks). A technique pioneered at Sun for the "Self" language (and then implemented for Strongtalk and then brought to Java) was "adaptive optimization and deoptimization" and was the Phd topic of Urs Hoelzle (Google's VP of Engineering). One of the key aspects is inlining across method boundaries as this removes method look-up, call stack handling and opens the way for code optimization across method boundaries (at the cost of RAM usage).

In OpenJDK, V8 and JavaScriptCore this adaptive optimization is typically implemented in C++ and requires quite some code. The code is complicated as it needs to optimize but also need to return to a basic function (deoptimize, e.g. if a method changed or the types passed don't match anymore), e.g. in the middle of a for loop with tons of inlined code (think of Array.map being inlined but then need to be de-inlined). A nice and long blog post of JSC can be found here describing the On Stack Replacement (OSR).

Long introduction and now to the new thing. In the OpensmalltalkVM a new approach called Sista has been picked and I find it is genius. Like with many problems the point of view and approach really matters. Instead of writing a lot of code in the VM the optimizer runs next to the application code. The key parts seem to be:

The revelation is the last part. By just optimizing from bytecode to bytecode the VM remains in charge of creating and managing machine code. The next part is that tooling in the higher language is better or at least the roundtrip is more quick (edit code and just compile the new method instead of running make, c++, ld). The productivity thanks to the abstraction and tooling is likely higher.

As last part the OSR is easier as well. In Smalltalk thisContext (the current stack frame, activation record) is an object as well. At the right point (when the JIT has either written back variables from register to the stack or at least knows where the value is) one can just manipulate thisContext, create and link news ones and then resume execution without all the magic in other VMs.

Go, Go and escape analysis

Ken Thompson and Robert Pike are well known persons and their Go programming language is a very interesting system programming language. Like with all new languages I try to get real world experience with the language, the tooling and which kind of problems can be solved with it. I have debugged and patched some bigger projects and written two small applications with it.

There is plenty I like. The escape analysis of the compiler is fun (especially now that I know it was translated from the Plan9 C compiler from C to Go), the concurrency model is good (though allowing shared state), the module system makes sense (but makes forking harder than necessary), being able to cross compile to any target from any system.

Knowing a bit of Erlang (and continuing to read the Phd Thesis of Joe Armstrong) and being a heavy Smalltalk user there are plenty of things missing. It starts with vague runtime error messages (e.g. panicslice not having parameters) and goes to runtime and post-runtime inspection. In Smalltalk thanks to the abstraction a lot of hard things are easy and I would have wished for some of them to be in Go. Serialize all unrecovered panics? Debugging someone else's code seems like pre 1980…

So for many developers Go is a big improvement but for some people with a wider view it might look like a lost opportunity. But that can only be felt by developers that have experienced higher abstraction and productivity.

Unsupervised machine learning

but that is for another dump…

25 Sep 2017 10:11am GMT

02 Sep 2017

feedPlanet Openmoko

Harald "LaF0rge" Welte: Purism Librem 5 campaign

There's a new project currently undergoing crowd funding that might be of interest to the former Openmoko community: The Purism Librem 5 campaign.

Similar to Openmoko a decade ago, they are aiming to build a FOSS based smartphone built on GNU/Linux without any proprietary drivers/blobs on the application processor, from bootloader to userspace.

Furthermore (just like Openmoko) the baseband processor is fully isolated, with no shared memory and with the Linux-running application processor being in full control.

They go beyond what we wanted to do at Openmoko in offering hardware kill switches for camera/phone/baseband/bluetooth. During Openmoko days we assumed it is sufficient to simply control all those bits from the trusted Linux domain, but of course once that might be compromised, a physical kill switch provides a completely different level of security.

I wish them all the best, and hope they can leave a better track record than Openmoko. Sure, we sold some thousands of phones, but the company quickly died, and the state of software was far from end-user-ready. I think the primary obstacles/complexities are verification of the hardware design as well as the software stack all the way up to the UI.

The budget of ~ 1.5 million seems extremely tight from my point of view, but then I have no information about how much Puri.sm is able to invest from other sources outside of the campaign.

If you're a FOSS developer with a strong interest in a Free/Open privacy-first smartphone, please note that they have several job openings, from Kernel Developer to OS Developer to UI Developer. I'd love to see some talents at work in that area.

It's a bit of a pity that almost all of the actual technical details are unspecified at this point (except RAM/flash/main-cpu). No details on the cellular modem/chipset used, no details on the camera, neither on the bluetooth chipset, wifi chipset, etc. This might be an indication of the early stage of their plannings. I would have expected that one has ironed out those questions before looking for funding - but then, it's their campaign and they can run it as they see it fit!

I for my part have just put in a pledge for one phone. Let's see what will come of it. In case you feel motivated by this post to join in: Please keep in mind that any crowdfunding campaign bears significant financial risks. So please make sure you made up your mind and don't blame my blog post for luring you into spending money :)

02 Sep 2017 10:00pm GMT

01 Sep 2017

feedPlanet Openmoko

Harald "LaF0rge" Welte: First actual XMOS / XCORE project

For many years I've been fascinated by the XMOS XCore architecture. It offers a surprisingly refreshing alternative virtually any other classic microcontroller architectures out there. However, despite reading a lot about it years ago, being fascinated by it, and even giving a short informal presentation about it once, I've so far never used it. Too much "real" work imposes a high barrier to spending time learning about new architectures, languages, toolchains and the like.

Introduction into XCore

Rather than having lots of fixed-purpose built-in "hard core" peripherals for interfaces such as SPI, I2C, I2S, etc. the XCore controllers have a combination of

  • I/O ports for 1/4/8/16/32 bit wide signals, with SERDES, FIFO, hardware strobe generation, etc
  • Clock blocks for using/dividing internal or external clocks
  • hardware multi-threading that presents 8 logical threads on each core
  • xCONNECT links that can be used to connect multiple processors over 2 or 5 wires per direction
  • channels as a means of communication (similar to sockets) between threads, whether on the same xCORE or a remote core via xCONNECT
  • an extended C (xC) programming language to make use of parallelism, channels and the I/O ports

In spirit, it is like a 21st century implementation of some of the concepts established first with Transputers.

My main interest in xMOS has been the flexibility that you get in implementing not-so-standard electronics interfaces. For regular I2C, UART, SPI, etc. there is of course no such need. But every so often one encounters some interface that's very rately found (like the output of an E1/T1 Line Interface Unit).

Also, quite often I run into use cases where it's simply impossible to find a microcontroller with a sufficient number of the related peripherals built-in. Try finding a microcontroller with 8 UARTs, for example. Or one with four different PCM/I2S interfaces, which all can run in different clock domains.

The existing options of solving such problems basically boil down to either implementing it in hard-wired logic (unrealistic, complex, expensive) or going to programmable logic with CPLD or FPGAs. While the latter is certainly also quite interesting, the learning curve is steep, the tools anything but easy to use and the synthesising time (and thus development cycles) long. Furthermore, your board design will be more complex as you have that FPGA/CPLD and a microcontroller, need to interface the two, etc (yes, in high-end use cases there's the Zynq, but I'm thinking of several orders of magnitude less complex designs).

Of course one can also take a "pure software" approach and go for high-speed bit-banging. There are some ARM SoCs that can toggle their pins. People have reported rates like 14 MHz being possible on a Raspberry Pi. However, when running a general-purpose OS in parallel, this kind of speed is hard to do reliably over long term, and the related software implementations are going to be anything but nice to write.

So the XCore is looking like a nice alternative for a lot of those use cases. Where you want a microcontroller with more programmability in terms of its I/O capabilities, but not go as far as to go full-on with FPGA/CPLD development in Verilog or VHDL.

My current use case

My current use case is to implement a board that can accept four independent PCM inputs (all in slave mode, i.e. clock provided by external master) and present them via USB to a host PC. The final goal is to have a board that can be combined with the sysmoQMOD and which can interface the PCM audio of four cellular modems concurrently.

While XMOS is quite strong in the Audio field and you can find existing examples and app notes for I2S and S/PDIF, I couldn't find any existing code for a PCM slave of the given requirements (short frame sync, 8kHz sample rate, 16bit samples, 2.048 MHz bit clock, MSB first).

I wanted to get a feeling how well one can implement the related PCM slave. In order to test the slave, I decided to develop the matching PCM master and run the two against each other. Despite having never written any code for XMOS before, nor having used any of the toolchain, I was able to implement the PCM master and PCM slave within something like ~6 hours, including simulation and verification. Sure, one can certainly do that in much less time, but only once you're familiar with the tools, programming environment, language, etc. I think it's not bad.

The biggest problem was that the clock phase for a clocked output port cannot be configured, i.e. the XCore insists on always clocking out a new bit at the falling edge, while my use case of course required the opposite: Clocking oout new signals at the rising edge. I had to use a second clock block to generate the inverted clock in order to achieve that goal.

Beyond that 4xPCM use case, I also have other ideas like finally putting the osmo-e1-xcvr to use by combining it with an XMOS device to build a portable E1-to-USB adapter. I have no clue if and when I'll find time for that, but if somebody wants to join in: Let me know!

The good parts

Documentation excellent

I found the various pieces of documentation extremely useful and very well written.

Fast progress

I was able to make fast progress in solving the first task using the XMOS / Xcore approach.

Soft Cores developed in public, with commit log

You can find plenty of soft cores that XMOS has been developing on github at https://github.com/xcore, including the full commit history.

This type of development is a big improvement over what most vendors of smaller microcontrollers like Atmel are doing (infrequent tar-ball code-drops without commit history). And in the case of the classic uC vendors, we're talking about drivers only. In the XMOS case it's about the entire logic of the peripheral!

You can for example see that for their I2C core, the very active commit history goes back to January 2011.

xSIM simulation extremely helpful

The xTIMEcomposer IDE (based on Eclipse) contains extensive tracing support and an extensible near cycle accurate simulator (xSIM). I've implemented a PCM mater and PCM slave in xC and was able to simulate the program while looking at the waveforms of the logic signals between those two.

The bad parts

Unfortunately, my extremely enthusiastic reception of XMOS has suffered quite a bit over time. Let me explain why:

Hard to get XCore chips

While the product portfolio on on the xMOS website looks extremely comprehensive, the vast majority of the parts is not available from stock at distributors. You won't even get samples, and lead times are 12 weeks (!). If you check at digikey, they have listed a total of 302 different XMOS controllers, but only 35 of them are in stock. USB capable are 15. With other distributors like Farnell it's even worse.

I've seen this with other semiconductor vendors before, but never to such a large extent. Sure, some packages/configurations are not standard products, but having only 11% of the portfolio actually available is pretty bad.

In such situations, where it's difficult to convince distributors to stock parts, it would be a good idea for XMOS to stock parts themselves and provide samples / low quantities directly. Not everyone is able to order large trays and/or capable to wait 12 weeks, especially during the R&D phase of a board.

Extremely limited number of single-bit ports

In the smaller / lower pin-count parts, like the XU[F]-208 series in QFN/LQFP-64, the number of usable, exposed single-bit ports is ridiculously low. Out of the total 33 I/O lines available, only 7 can be used as single-bit I/O ports. All other lines can only be used for 4-, 8-, or 16-bit ports. If you're dealing primarily with serial interfaces like I2C, SPI, I2S, UART/USART and the like, those parallel ports are of no use, and you have to go for a mechanically much larger part (like XU[F]-216 in TQFP-128) in order to have a decent number of single-bit ports exposed. Those parts also come with twice the number of cores, memory, etc- which you don't need for slow-speed serial interfaces...

Insufficient number of exposed xLINKs

The smaller parts like XU[F]-208 only have one xLINK exposed. Of what use is that? If you don't have at least two links available, you cannot daisy-chain them to each other, let alone build more complex structures like cubes (at least 3 xLINKs).

So once again you have to go to much larger packages, where you will not use 80% of the pins or resources, just to get the required number of xLINKs for interconnection.

Change to a non-FOSS License

XMOS deserved a lot of praise for releasing all their soft IP cores as Free / Open Source Software on github at https://github.com/xcore. The License has basically been a 3-clause BSD license. This was a good move, as it meant that anyone could create derivative versions, whether proprietary or FOSS, and there would be virtually no license incompatibilities with whatever code people wanted to write.

However, to my very big disappointment, more recently XMOS seems to have changed their policy on this. New soft cores (released at https://github.com/xmos as opposed to the old https://github.com/xcore) are made available under a non-free license. This license is nothing like BSD 3-clause license or any other Free Software or Open Source license. It restricts the license to use the code together with an XMOS product, requires the user to contribute fixes back to XMOS and contains references to importand export control. This license is incopatible with probably any FOSS license in existance, making it impossible to write FOSS code on XMOS while using any of the new soft cores released by XMOS.

But even beyond that license change, not even all code is provided in source code format anymore. The new USB library (lib_usb) is provided as binary-only library, for example.

If you know anyone at XMOS management or XMOS legal with whom I could raise this topic of license change when transitioning from older sc_* software to later lib_* code, I would appreciate this a lot.

Proprietary Compiler

While a lot of the toolchain and IDE is based on open source (Eclipse, LLVM, ...), the actual xC compiler is proprietary.

Further Reading

01 Sep 2017 10:00pm GMT

12 Nov 2011

feedPlanet Linux-to-go

Paul 'pfalcon' Sokolovsky: Shopping for 3D TV...

Shopping for 3D TV (again), few findings:

12 Nov 2011 6:55pm GMT

Paul 'pfalcon' Sokolovsky: Hacking Luxeon SP-1

I finally going to get Arduino, and while I'm choosing flavor and waiting for it, I can't help but disassembling all devices I have at home, each time speaking: "This must have Arduino inside!" (meaning of course that I expect it to be based on general-purpose MCU). Gosh, I usually get "blob chip" (uncased chip with blob of epoxy on top).

Well, I finally had my expectations fulfilled - Luxeon SP-1 voltage stabilizer/cutter features ATMEGA48V-10PU (Flash: 4k, EEPROM: 256, RAM:512). Not only that, it is installed in DIP socket! Buy from Luxeon, they're hacker-friendly ;-).

I bought the device actually for a wattmeter it features (which fact is hard to figure out from common specs found in the shops, I accidentally read somebody mentioning it on a forum). The wattmeter is of course not bright - for a lamp rated 100W it shows 88W, and for more powerful equipment (like perforator) understates wattage even more (maybe it's difference between real and apparent power factor).

Still, for $17 you get Arudino-alike with voltage/current sensor and hacking possibility. Woot!

BOM:
High-power board:

MCU board:


12 Nov 2011 5:58pm GMT

10 Nov 2011

feedPlanet Linux-to-go

Paul 'pfalcon' Sokolovsky: Links for November 2011

Kindle:


Linux kernel module tricks:

10 Nov 2011 3:21pm GMT

feedAndroid Forums

Latest action game INC from OrangePixel now available!

From the developer of Meganoid and Stardash comes a new action arcade game: INC! http://www.youtube.com/watch?v=9j5OEG-3RyM Get it from the...

10 Nov 2011 9:31am GMT

Free online video chat

More than 1000 broadcast cameras for you online - the most incendiary models in Russia. 1000 girls, 1000, the temptations, 1000, full of desire - all...

10 Nov 2011 7:48am GMT

Layout problem

Hi Friends I decided to work with a tab layout application. Program consist of 3 tabs and a button. I like to place the button below the tab. ...

10 Nov 2011 5:20am GMT

19 Oct 2011

feedPlanet OpenEZX

Antonio Ospite: Gnome 3: go to Shell? Not just yet, thanks.

In Debian Unstable the transition to Gnome 3 is taking place; when Gnome 3.0 firstly came out some unnamed geeky users complained loudly about the design decisions of the development team to push strongly towards gnome-shell as a new default UI; gnome-shell was designed focusing on usability (usability is a metric relative to a certain target audience BTW) and simplicity, hiding a lot of details from the users. Obviously you can never make everyone happy so some of us simply happened to be "out of target": you know us computer people (*cough cough*), we like to be in charge and control The Machine... I must admit I still don't have a definitive opinion about the gnome-shell concept, for now I just know that it does not suit me; I am going to try it eventually, maybe I'll get used to it, but in the mean time I need my desktop back just like I shaped it through the years; can this be done without loosing all the good Gnome technologies (Empathy over all of them)?

To be completely fair I have to say that there is little to complain about with Gnome developers, we can still get our good old GNOME desktop fully back by using the fall-back mode based on gnome-panel and live happily ever after, let's take a look at how this can be accomplished.

NOTE: GNOME people state that the fall-back mode is meant for systems with older graphic cards which cannot run gnome-shell, however it can very well be seen as a good opportunity for those who do not want to run gnome-shell just yet.

Getting back to the topic: some minor touches are needed to make the panel look more like what we are used to, maybe some of these settings could even become default for fall-back mode, we'll see.

First, enable fall-back mode (on Debian there is a dedicated session you can choose from the Log-in Manager for that) and change some desktop settings, in a terminal type:

$ gsettings set org.gnome.desktop.session session-name 'gnome-fallback'
$ gsettings set org.gnome.desktop.interface 'menus-have-icons' true
$ gsettings set org.gnome.desktop.interface 'buttons-have-icons' true
$ gsettings set org.gnome.desktop.background 'show-desktop-icons' true

gnome-tweak-tool can be used for some of these settings like shown in the attached images.

Then rearrange the applets on the panel as you please (use Alt-RightClick to access the panel properties), and fix the theming using this patch to have a light panel again (against gnome-themes-standard=3.0.2-1):

$ mkdir $HOME/.themes
$ cd $HOME/.themes
$ cp -r /usr/share/themes/Adwaita Adwaita-fallback
$ cd Adwaita-fallback
$ patch -p1 < $HOME/adwaita-fallback-panel-theme.patch
$ gsettings set org.gnome.desktop.interface 'gtk-theme' 'Adwaita-fallback'

Some final touches for the Metacity window manager and to the clock applet, and we are all set:

$ gconftool-2 --type string --set /apps/metacity/general/focus_mode mouse
$ gconftool-2 --type boolean --set /apps/metacity/general/compositing_manager true
$ gconftool-2 --type string --set /apps/panel3-applets/clock/custom_format '<span color="#333">%a %d %b</span> <b>%H:%M</b>'
$ gconftool-2 --type string --set /apps/panel3-applets/clock/format custom

Ah, in the new gnome-panel based on Gtk3 there are still some details to take care of, I hope issues like that will be addressed and that the panel will be supported for quite some time.

Attached images:
Gnome Shell default look on Debian
gnome-tweak-tool show desktop icons
Gnome 3 fall-back mode default look on Debian
Gnome 3 fall-back mode applets rearranged
Gnome 3 fall-back mode rethemed to have a light panel
Attached files:
text/x-diff iconAdwaita theme patch for fall-back mode

19 Oct 2011 9:37pm GMT

09 Jun 2011

feedPlanet OpenEZX

Michael Lauer: The Eagle Has Landed!

After letting us wait for a bit longer than scheduled (13 days), the hospital initiated the contractions. For the first couple of hours, everything went just perfect, but then the little one got stuck on the way and we had to resort to a cesarean section. Lara Marie Lauer was born 8th of June at 04:41 (AM) with 3460 gramms and 49 cm.

Mummy was still on intensive care and so they gave her to me. I can't express the feelings I had in this very moment. I'm still kind of overwhelmed every time I see her. Thanks for all of you who waited anxiously with me and those who prayed for us. The most important tasks for the near future is getting Mummy to recover and Lara Marie to become accustomed to us and the rest of the outside world.

Please bear with me if in the next time I'm not as responsive as usually :)

Lara Marie Lauer

09 Jun 2011 4:06pm GMT

30 May 2011

feedPlanet OpenEZX

Michael Lauer: German Post on time!

And now for something completely different… while we are all waiting for my baby to arrive (who was scheduled for 25th of May), she just received her first greeting card - together with a personalized bib and a towel (with integrated hood - pretty fancy!) from my good friends at #openmoko-cdevel.

Guys, seeing this card was very heartwarming - it means a lot to me that you share my anticipation, thanks a lot! And I'm 100% sure she will appreciate her gifts… now let's cross fingers it won't take much longer… waiting is the hardest part of it :)

Yours,

Mickey.

30 May 2011 8:54am GMT