18 Aug 2016

feedPlanet Maemo

2016-07-26 Meeting Minutes

Meeting held 2016-07-26 on FreeNode, channel #maemo-meeting (logs)

Attending: pichlo, Win7Mac, eekkelund, reinob, juiceme

Partial: chem|st

Absent:

Summary of topics (ordered by discussion):


(Topic Approve pending project request on https://garage.maemo.org/admin/approve-pending.php):

(Topic Coding Competition):

(Topic Twitter account):

(Topic MC e.V.):


Action Items:
  • old items:
    • Coding competition planning (eekkelund)
    • The next GA meeting should be announced soon.
  • new items:
    • Set up ftp/sftp site for CC (reinob)
    • Create twitter account for maemo community(eekkelund)


Solved Action Items:

Find out if https is doable 0 Add to favourites0 Bury

18 Aug 2016 6:39am GMT

2016-07-19 Meeting Minutes

Meeting held 2016-07-19 on FreeNode, channel #maemo-meeting (logs)

Attending: Win7Mac, eekkelund, reinob

Partial:

Absent: pichlo, juiceme

Summary of topics (ordered by discussion):


(Topic Coding Competition):


Action Items:
  • old items:
    • Coding competition planning (eekkelund)
    • The next GA meeting should be announced soon.
  • new items:


Solved Action Items:

Find out if https is doable 0 Add to favourites0 Bury

18 Aug 2016 6:38am GMT

2016-07-12 Meeting Minutes

Meeting held 2016-07-12 on FreeNode, channel #maemo-meeting (logs)

Attending: eekkelund, pichlo, reinob

Partial:

Absent: Win7Mac, juiceme

Summary of topics (ordered by discussion):


(Topic Coding Competition):


Action Items:
  • old items:
    • Coding competition planning (eekkelund)
    • The next GA meeting should be announced soon.
  • new items:


Solved Action Items:

Find out if https is doable 0 Add to favourites0 Bury

18 Aug 2016 6:36am GMT

2016-07-05 Meeting Minutes

Meeting held 2016-07-05 on FreeNode, channel #maemo-meeting (logs)

Attending: juiceme, eekkelund, pichlo, Win7Mac, reinob

Partial:

Absent:

Summary of topics (ordered by discussion):


(Topic read-only option for TOR endpoints):

(Topic Coding Competition):


Action Items:
  • old items:
    • The next GA meeting should be announced soon.
  • new items:
    • Coding competition planning (eekkelund)


Solved Action Items:

Find out if https is doable 0 Add to favourites0 Bury

18 Aug 2016 6:35am GMT

05 Aug 2016

feedPlanet Maemo

On OGRE versions

Currently one can choose between the following OGRE versions
1.9, 1.10, 2.0 and 2.1

However the versioning scheme has become completely arbitrary while still resembling semantic versioning.
As a consequence somebody even had to put a "What version to choose?" guide on the OGRE homepage.

Unfortunately the guide confuses more than it helps:

You might wonder why 2.1 while having the latest features, lacks some bugfixes of older releases. Read on..

Lack of branching concept

Usually projects have a default master or trunk branch where all development eventually ends up and optionally some maintenance and bleeding edge branches. See this for a small overview of the according workflows.

Now OGRE has the following branches: v1-9, v1-10, v2-0, v2-1 and default. By common sense default would contain the most recent code while the other branches would be maintenance branches of the respective releases. But not today! Actually all of these branches are under active development with 2.1 containing the latest features and default representing the current stable version. (actually default and 1.10 are pretty much the same)

As there is no clear code flow between the branches this means that branches are diverging. Fixes for issues in one branch do not distribute across the others. Examples:

As you can see we got all possible directions here. The situation can not be easily resolved by successive merging branches into each other.

With the github fork, I therefore manually compared the different branches and cherry-picked all fixed into master (v1-10).

This is why I initially called it the stable fork. Even though it contains new features as discussed in my last post, the API is backwards compatible to existing code/ add-ons and it aggregates all bugfixes in one branch.

0 Add to favourites0 Bury

05 Aug 2016 12:24pm GMT

30 Jul 2016

feedPlanet Maemo

Maemo Community e.V. - Invitation to the General Assembly 2016

Maemo Community e.V. - Einladung zur Mitgliederversammlung 2016

Sehr geehrtes Mitglied, unsere diesjährige ordentliche Mitgliederversammlung findet am 04.09.2016 16:00 UTC im IRC Freenode Channel #maemo-meeting statt.

Auf der Tagesordnung stehen folgende Themen:
1. Begrüßung durch die Vorsitzende des Vorstands
2. Feststellung der ordnungsgemäßen Einberufung und der Beschlussfähigkeit der Mitgliederversammlung
3. Entgegennahme des Jahresberichts für das abgelaufene Geschäftsjahr und Entlastung des Vorstands
4. Neuwahl des Vorstandes
5. Änderung der Satzung §7 und §8, und der Vereinsordnung §1.1, §3.2 und §4.1. Erläuterung im Anhang.
6. Verschiedenes

Anträge auf Ergänzungen der Tagesordnung müssen bis eine Woche vor der Versammlung schriftlich beim Vorstand eingereicht werden (§ 9 Abs. 2 der Satzung).

Mit freundlichen Grüßen
Win7Mac/Gido Griese

############################ English##########################

Maemo Community e.V. - Invitation to the General Assembly 2016

Dear Member, our Annual General Meeting will be held on Sunday, 2016-09-04 at 16:00 UTC on IRC Freenode channel #maemo-meeting.

Unless any further issues are raised, the agenda includes the following topics:
1. Welcome by the Chairman of the Board
2. Determination of the proper convocation and the quorum of the General Assembly
3. Acceptance of the annual report for the fiscal year and actions of the Executive
4. Election of the Board Directors
5. Amendment of Articles of Association §7 and §8, Association Rules §1.1, §3.2 and $4.1 - please see appendix for details [1].
6. Any other business

Requests for additions to the agenda must be submitted to the Board in writing one week prior to the meeting (§ 9 para. 2 of the Statutes).

Yours sincerely
Win7Mac/Gido Griese


[1]http://maemo.org/community/council/referendum_2015/


Anhang / APPENDIX

* Änderungen der Satzung/Amendments to the Articles of Association:
§7 Vorstand (Board of directors)
(5) Der Vorstand führt die Beschlüsse der aktiven Mitgliederversammlung und der des Vorstandsrats aus, sofern dem Vorstandsrat diese Entscheidungshoheit laut Satzung oder Vereinsordnung eingeräumt wird.
(5) The Board of Directors carries out the resolutions of the General Assembly and of the Council in so far as this decision's jurisdiction is delegated to Council according to the statutes.

§8 Vorstandsrat (Council)
(4) Aufgaben des Vorstandsrats sowie Regelungen zu dessen Arbeit, inklusive der Einberufung von Sitzungen, deren Ablauf und die Durchführung von Abstimmungen, sind in der Vereinsordnung festgelegt.
(4) The council's duties, rules to announce meetings, their proceedings and executions of votes are regulated by the Association Rules.

* Änderungen der Vereinsordnung/Amendments to the Association Rules
§ 1.1
(2) § 4.1 (Vorstandsratswahlen) dieser Vereinsordnung kann durch ein Referendum der passiven Mitgliederversammlung geändert oder aufgehoben werden. Änderungen dieser Wahlordnung durch die aktive Mitgliederversammlung müssen durch Beschluss der passiven Mitgliederversammlung bestätigt werden (Referendum). Sollte die Änderung von der passiven Mitgliederversammlung abgelehnt werden, kann der Änderungsvorschlag überarbeitet und erneut zur Abstimmung gebracht werden.

(2) § 4.1 (Council election rules) of these association rules can be changed or dismissed with a referendum by the Passive Members' Meeting. Changes to these election rules by the General Assembly must be confirmed by a resolution of the passive members meeting (referendum). Should the amendment be rejected by the passive members meeting, the proposed amendment can be revised and brought again to the vote.

§3.2 Vorstandsrat (council)

(7) Aufgaben und Zuständigkeiten des Vorstandsrats sind wie folgt:
a) Vertretung der assoziierten Mitglieder und ihrer Interessen
b) Beratung des Vorstands in allgemeinen Belangen
c) Der Vorstandsrat ist für die Durchführung aller Wahlen zuständig

(7) Tasks and competences of the Council are as follows:
a) Representation of the associated members (garage account) and their interests.
b) Counseling of the Board of directors in general matters
c) Council is responsible to execute all elections

§ 4.1 Vorstandsratswahlen (Council Elections)

(6) Kandidaten mit einem professionellen bzw. witschaftlichen Interesse in Maemo, z.B. Mitarbeiter von Firmen, die in Bezug zu Maemo Hard- oder Software stehen, müssen ihr Interesse bei der Nominierung erklären. Bleibt dies aus, kann der Vorstandsrat die Kandidatur des Mitglieds für ungültig erklären und somit ausschliessen.

(6) Nominees with a professional interest in Maemo, such as working for a company involved in Maemo-related software development - must declare their interest when advertising their nomination. Failure to do so may result in the Board of Directors, or the outgoing Council, declaring their nomination invalid and so bar them from standing in the current election.

(9) Das Datum der nächsten Wahl soll mindestens 4 Wochen vorher auf maemo.org bekannt gegeben werden.

(9) The election date must be publicised at least 4 weeks in advance of the election.

(11) Sollten nach Ende der Nominierungsphase weniger als 4 Kandidaten zur Wahl stehen, kann die Wahl nicht abgehalten werden.
a) Die Nominierungsphase wird dann um 4 Wochen verlängert und die Wahl entsprechend verschoben.
b) Sollten nach der verlängerten Nominierungsphase weniger als 4 Kandidaten zur Wahl stehen, bestimmt der scheidende Vorstandsrat zusammen mit dem Vorstand das weitere Vorgehen. Eine Lösung, die zu weiteren Kandidaten und einer ordentlichen Wahl führt, soll dabei bevorzugt werden, andere Möglichkeiten werden nicht ausgeschlossen.

(11) If there are less than 4 candidates when the nominations close, the election can't be held.
a) The nomination period will be extended by 4 weeks and the election postponed similarly.
b) If, after 4 weeks's delay, there are still fewer than 4 candidates; the outgoing council will decide - in conjunction with Board of Directors - about how to proceed. The preferred solution is to encourage more members of the maemo.org community to participate and re-run the election, but all options are open.

(12) Änderungen der Vorstandsratswahlen müssen durch Beschluss durch die aktive Mitgliederversammlung oder die passive Mitgliederversammlung bestätigt werden.
a) Die Teilnahme an Referenda unterliegt den gleichen Bestimmungen, die für Vorstandsratswahlen gelten.
b) Bevor ein Referendum zur Abstimmung gelangen kann, muss ein entsprechender Entwurf für mindestens 4 Wochen zur Diskussion stehen.
c) Der Zeitraum, in welchem über das Referendum abgestimmt werden kann, soll genauso lang sein, wie bei regulären Vorstandsratswahlen.

(12)Changes to any of the above rules must be approved by a resolution of the Passive Members' Meeting (community referendum) or General Assembly.
a) Voting in such, will be open to anyone eligible to vote in the respective meeting.
b) The referendum options must be debated for a minimum of 4 weeks prior to the referendum.
c) Referendum voting will be open for the same length of time as the council elections.1 Add to favourites0 Bury

30 Jul 2016 4:11pm GMT

28 Jul 2016

feedPlanet Maemo

Truly huge files and the problem of continuous virtual address space

As we all know does mmap, or even worse on Windows CreateFileMapping, need contiguous virtual address space for a given mapping size. That can become a problem when you want to load a file of a gigabyte with mmap.

The solution is of course to mmap the big file using multiple mappings. For example like adapting yesterday's demo this way:

void FileModel::setFileName(const QString &fileName)
{
    ...
    if (m_file->open(QIODevice::ReadOnly)) {
        if (m_file->size() > MAX_MAP_SIZE) {
            m_mapSize = MAX_MAP_SIZE;
            m_file_maps.resize(1 + m_file->size() / MAX_MAP_SIZE, nullptr);
        } else {
            m_mapSize = static_cast(m_file->size());
            m_file_maps.resize(1, nullptr);
        }
        ...
    } else {
        m_index->open(QFile::ReadOnly);
        m_rowCount = m_index->size() / 4;
    }
    m_file_maps[0] = m_file->map(0, m_mapSize, QFileDevice::NoOptions);
    qDebug() << "Done loading " << m_rowCount << " lines";
    map_index = m_index->map(0, m_index->size(), QFileDevice::NoOptions);

    beginResetModel();
    endResetModel();
    emit fileNameChanged();
}

And in the data() function:

QVariant FileModel::data( const QModelIndex& index, int role ) const
{
    QVariant ret;
    ...
    quint32 mapIndex = pos_i / MAX_MAP_SIZE;
    quint32 map_pos_i = pos_i % MAX_MAP_SIZE;
    quint32 map_end_i = end_i % MAX_MAP_SIZE;
    uchar* map_file = m_file_maps[mapIndex];
    if (map_file == nullptr)
        map_file = m_file_maps[mapIndex] = m_file->map(mapIndex * m_mapSize, m_mapSize, QFileDevice::NoOptions);
    position = m_file_maps[mapIndex] + map_pos_i;
    if (position) {
            const int length = static_cast(end_i - pos_i);
            char *buffer = (char*) alloca(length+1);
            if (map_end_i >= map_pos_i)
                strncpy (buffer, (char*) position, length);
            else {
                const uchar *position2 = m_file_maps[mapIndex+1];
                if (position2 == nullptr) {
                    position2 = m_file_maps[mapIndex+1] = m_file->map((mapIndex+1) *
                         m_mapSize, m_mapSize, QFileDevice::NoOptions);
                }
                strncpy (buffer, (char*) position, MAX_MAP_SIZE - map_pos_i);
                strncpy (buffer + (MAX_MAP_SIZE - map_pos_i), (char*) position2, map_end_i);
            }
            buffer[length] = 0;
            ret = QVariant(QString(buffer));
        }
    }
    return ret;
}

You could also not use mmap for the very big source text file and use m_file.seek(map_pos_i) and m_file.read(buffer, length). The most important mapping is of course the index one, as the reading of the individual lines can also be done fast enough with normal read() calls (as long as you don't have to do it for each and every line of the very big file and as long as you know in a O(1) way where the QAbstractListModel's index.row()'s data is).

But you already knew that. Right?

0 Add to favourites0 Bury

28 Jul 2016 12:42pm GMT

26 Jul 2016

feedPlanet Maemo

Loading truly truly huge text files with a QAbstractListModel

Sometimes people want to do crazy stuff like loading a gigabyte sized plain text file into a Qt view that can handle QAbstractListModel. Like for example a QML ListView. You know, the kind of files you generate with this commando:

base64 /dev/urandom | head -c 100000000 > /tmp/file.txt

But, how do they do it?

FileModel.h

So we will make a custom QAbstractListModel. Its private member fields I will explain later:

#ifndef FILEMODEL_H
#define FILEMODEL_H

#include <QObject>
#include <QVariant>
#include <QAbstractListModel>
#include <QFile>

class FileModel: public QAbstractListModel {
    Q_OBJECT

    Q_PROPERTY(QString fileName READ fileName WRITE setFileName NOTIFY fileNameChanged )
public:
    explicit FileModel( QObject* a_parent = nullptr );
    virtual ~FileModel();

    int columnCount(const QModelIndex &parent) const;
    int rowCount( const QModelIndex& parent =  QModelIndex() ) const Q_DECL_OVERRIDE;
    QVariant data( const QModelIndex& index, int role = Qt::DisplayRole ) const  Q_DECL_OVERRIDE;
    QVariant headerData( int section, Qt::Orientation orientation,
                         int role = Qt::DisplayRole ) const  Q_DECL_OVERRIDE;
    void setFileName(const QString &fileName);
    QString fileName () const
        { return m_file->fileName(); }
signals:
    void fileNameChanged();
private:
    QFile *m_file, *m_index;
    uchar *map_file;
    uchar *map_index;
    int m_rowCount;
    void clear();
};

#endif// FILEMODEL_H

FileModel.cpp

We will basically scan the very big source text file for newline characters. We'll write the offsets of those to a file suffixed with ".mmap". We'll use that new file as a sort of "partition table" for the very big source text file, in the data() function of QAbstractListModel. But instead of sectors and files, it points to newlines.

The reason why the scanner itself isn't using the mmap's address space is because apparently reading blocks of 4kb is faster than reading each and every byte from the mmap in search of \n characters. Or at least on my hardware it was.

You should probably do the scanning in small qEventLoop iterations (make sure to use nonblocking reads, then) or in a thread, as your very big source text file can be on a unreliable or slow I/O device. Plus it's very big, else you wouldn't be doing this (please promise me to just read the entire text file in memory unless it's hundreds of megabytes in size: don't micro optimize your silly homework notepad.exe clone).

Note that this is demo code with a lot of bugs like not checking for \r and god knows what memory leaks and stuff was remaining when it suddenly worked. I leave it to the reader to improve this. An example is that you should check for validity of the ".mmap" file: your very big source text file might have changed since the newline partition table was made.

Knowing that I'll soon find this all over the place without any of its bugs fixed, here it comes ..

#include "FileModel.h"

#include <QDebug>

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <pthread.h>
#include <unistd.h>

FileModel::FileModel( QObject* a_parent )
    : QAbstractListModel( a_parent )
    , m_file (nullptr)
    , m_index(nullptr)
    , m_rowCount ( 0 ) { }

FileModel::~FileModel() { clear(); }

void FileModel::clear()
{
    if (m_file) {
        if (m_file->isOpen() && map_file != nullptr)
            m_file->unmap(map_file);
        delete m_file;
    }
    if (m_index) {
        if (m_index->isOpen() && map_index != nullptr)
            m_index->unmap(map_index);
        delete m_index;
    }
}

void FileModel::setFileName(const QString &fileName)
{
   clear();
   m_rowCount = 0;
   m_file = new QFile(fileName);
   int cur = 0;
   m_index = new QFile(m_file->fileName() + ".mmap");
   if (m_file->open(QIODevice::ReadOnly)) {
       if (!m_index->exists()) {
           char rbuffer[4096];
           m_index->open(QIODevice::WriteOnly);
           char nulbuffer[4];
           int idxnul = 0;
           memset( nulbuffer +0, idxnul >> 24 & 0xff, 1 );
           memset( nulbuffer +1, idxnul >> 16 & 0xff, 1 );
           memset( nulbuffer +2, idxnul >>  8 & 0xff, 1 );
           memset( nulbuffer +3, idxnul >>  0 & 0xff, 1 );
           m_index->write( nulbuffer, sizeof(quint32));
           qDebug() << "Indexing to" << m_index->fileName();
           while (!m_file->atEnd()) {
               int in = m_file->read(rbuffer, 4096);
               if (in == -1)
                   break;
               char *newline = (char*) 1;
               char *last = rbuffer;
               while (newline != 0) {
                   newline = strchr ( last, '\n');
                   if (newline != 0) {
                     char buffer[4];
                     int idx = cur + (newline - rbuffer);
                     memset( buffer +0, idx >> 24 & 0xff, 1 );
                     memset( buffer +1, idx >> 16 & 0xff, 1 );
                     memset( buffer +2, idx >>  8 & 0xff, 1 );
                     memset( buffer +3, idx >>  0 & 0xff, 1 );
                     m_index->write( buffer, sizeof(quint32));
                     m_rowCount++;
                     last = newline + 1;
                  }
               }
               cur += in;
           }
           m_index->close();
           m_index->open(QFile::ReadOnly);
           qDebug() << "done";
       } else {
           m_index->open(QFile::ReadOnly);
           m_rowCount = m_index->size() / 4;
       }
       map_file= m_file->map(0, m_file->size(), QFileDevice::NoOptions);
       qDebug() << "Done loading " << m_rowCount << " lines";
       map_index = m_index->map(0, m_index->size(), QFileDevice::NoOptions);
   }
   beginResetModel();
   endResetModel();
   emit fileNameChanged();
}

static quint32
read_uint32 (const quint8 *data)
{
    return data[0] << 24 |
           data[1] << 16 |
           data[2] << 8 |
           data[3];
}

int FileModel::rowCount( const QModelIndex& parent ) const
{
    Q_UNUSED( parent );
    return m_rowCount;
}

int FileModel::columnCount(const QModelIndex &parent) const
{
    Q_UNUSED( parent );
    return 1;
}

QVariant FileModel::data( const QModelIndex& index, int role ) const
{
    if( !index.isValid() )
        return QVariant();
    if (role == Qt::DisplayRole) {
        QVariant ret;
        quint32 pos_i = read_uint32(map_index + ( 4 * index.row() ) );
        quint32 end_i;
        if ( index.row() == m_rowCount-1 )
            end_i = m_file->size();
        else
            end_i = read_uint32(map_index + ( 4 * (index.row()+1) ) );
        uchar *position;
        position = map_file +  pos_i;
        uchar *end = map_file + end_i;
        int length = end - position;
        char *buffer = (char*) alloca(length +1);
        memset (buffer, 0, length+1);
        strncpy (buffer, (char*) position, length);
        ret = QVariant(QString(buffer));
        return ret;
    }
    return QVariant();
}

QVariant FileModel::headerData( int section, Qt::Orientation orientation, int role ) const
{
    Q_UNUSED(section);
    Q_UNUSED(orientation);
    if (role != Qt::DisplayRole)
           return QVariant();
    return QString("header");
}

main.cpp

#include <QGuiApplication>
#include <QQmlApplicationEngine>
#include <QtQml>// qmlRegisterType

#include "FileModel.h"

int main(int argc, char *argv[])
{
    QGuiApplication app(argc, argv);
    qmlRegisterType<FileModel>( "FileModel", 1, 0, "FileModel" );
    QQmlApplicationEngine engine;
    engine.load(QUrl(QStringLiteral("qrc:/main.qml")));
    return app.exec();
}

main.qml

import QtQuick 2.3
import QtQuick.Window 2.2
import FileModel 1.0

Window {
    visible: true

    FileModel { id: fileModel }
    ListView {
        id: list
        anchors.fill: parent
        delegate: Text { text: display }
        MouseArea {
            anchors.fill: parent
            onClicked: {
                list.model = fileModel
                fileModel.fileName = "/tmp/file.txt"
            }
        }
    }
}

profile.pro

TEMPLATE = app
QT += qml quick
CONFIG += c++11
SOURCES += main.cpp \
    FileModel.cpp
RESOURCES += qml.qrc
HEADERS += \
    FileModel.h

qml.qrc

<RCC>
    <qresource prefix="/">
        <file>main.qml</file>
    </qresource>
</RCC>

0 Add to favourites0 Bury

26 Jul 2016 7:15pm GMT

09 Jul 2016

feedPlanet Maemo

Creating PyGTK app snaps with snapcraft

Snap is a new packaging format introduced by Ubuntu as an successor to dpkg aka debian package. It offers sandboxing and transactional updates and thus is a competitor to the flatpak format and resembles docker images.

As with every new technology the weakest point of working with snaps is the documentation. Your best bet so far is the snappy-playpen repository.

There are also some rough edges regarding desktop integration and python interoperability, so this is what the post will be about.

I will introduce some quircks that were needed to get teatime running, which is written in Python3 and uses Unity and GTK3 via GObject introspection.

The most important thing to be aware of is that snaps are similar to containers in that each snap has its own rootfs and only restricted access outside of it. This is basically what the sandboxing is about.
However a typical desktop application needs to know quite a lot about the outside world:

To declare that we want to write to home, play back sound and use unity features we use the plugs keyword like

apps:
    teatime:
         # ...
         plugs: [unity7, home, pulseaudio]

However we must also tell our app to look for the according libraries inside its snap instead of the system paths. For this one must change quite a few environment variables manually. Fortunately Ubuntu provides wrapper scripts that take care of this for us. They are called desktop-launchers.

To use the launcher the configures the GTK3 environment we have to extend the teatime part like this:

parts:
    teatime:
         # ...
         after: [desktop/gtk3]

However the desktop-launch script currently misses the environment varibles for GI repository files. To fix this we must use this rather lengthy startup command

env GI_TYPELIB_PATH=$SNAP/usr/lib/girepository-1.0:$SNAP/usr/lib/x86_64-linux-gnu/girepository-1.0 desktop-launch $SNAP/usr/share/teatime/teatime.py

the bad part here is that we hard-code the architecture. The proper solution is to just fix the desktop-launch script, which I did here.

Update

The change was merged, so the startup command simpifies to

apps:
    teatime:
        command: desktop-launch $SNAP/usr/share/teatime/teatime.py
        # ...

You can see the full snapcraft.yml here.

After this teatime will start, but the paths still have to be fixed. Inside a snap "/" still refers to the system root, so all absolute paths must be prefixed with $SNAP.

Actually I think the design of flatpak is more elegant in this regard where "/" points to the local rootfs and one does not have to change absolute paths. To bring in the system parts flatpak uses bind mounts.

Conclusions

Once you get the hang of how snaps work, packaging becomes quite straightforward, however currently there are still some drawbacks

0 Add to favourites0 Bury

09 Jul 2016 1:45pm GMT

01 Jul 2016

feedPlanet Maemo

Composition and aggregation with QObject

Consider these rather simple relationships between classes

Continuing on this subject, here are some code examples.

Class1 & Class2: Composition
An instance of Class1 can not exist without an instance of Class2.

Example of composition is typically a Bicycle and its Wheels, Saddle and a HandleBar: without these the Bicycle is no longer a Bicycle but just a Frame.

It can no longer function as a Bicycle. Example of when you need to stop thinking about composition versus aggregation is whenever you say: without the other thing can't in our software the first thing work.

Note that you must consider this in the context of Class1. You use aggregation or composition based on how Class2 exists in relation to Class1.

Class1 with QScopedPointer:

#ifndef CLASS1_H
#define CLASS1_H

#include <QObject>
#include <QScopedPointer>
#include <Class2.h>

class Class1: public QObject
{
    Q_PROPERTY( Class2* class2 READ class2 WRITE setClass2 NOTIFY class2Changed)
public:
    Class1( QObject *a_parent = nullptr )
        : QObject ( a_parent) {
        // Don't use QObject parenting on top here
        m_class2.reset (new Class2() );
    }
    Class2* class2() {
        return m_class2.data();
    }
    void setClass2 ( Class2 *a_class2 ) {
        Q_ASSERT (a_class2 != nullptr); // Composition can't set a nullptr!
        if ( m_class2.data() != a_class2 ) {
            m_class2.reset( a_class2 );
            emit class2Changed()
        }
    }
signals:
    void class2Changed();
private:
    QScopedPointer<Class2> m_class2;
};

#endif// CLASS1_H

Class1 with QObject parenting:

#ifndef CLASS1_H
#define CLASS1_H

#include <QObject>
#include <Class2.h>

class Class1: public QObject
{
    Q_PROPERTY( Class2* class2 READ class2 WRITE setClass2 NOTIFY class2Changed)
public:
    Class1( QObject *a_parent = nullptr )
        : QObject ( a_parent )
        , m_class2 ( nullptr ) {
        // Make sure to use QObject parenting here
        m_class2 = new Class2( this );
    }
    Class2* class2() {
        return m_class2;
    }
    void setClass2 ( Class2 *a_class2 ) {
         Q_ASSERT (a_class2 != nullptr); // Composition can't set a nullptr!
         if ( m_class2 != a_class2 ) {
             // Make sure to use QObject parenting here
             a_class2->setParent ( this );
             delete m_class2; // Composition can never be nullptr
             m_class2 = a_class2;
             emit class2Changed();
         }
    }
signals:
    void class2Changed();
private:
    Class2 *m_class2;
};

#endif// CLASS1_H

Class1 with RAII:

#ifndef CLASS1_H
#define CLASS1_H

#include <QObject>
#include <QScopedPointer>

#include <Class2.h>

class Class1: public QObject
{
    Q_PROPERTY( Class2* class2 READ class2 CONSTANT)
public:
    Class1( QObject *a_parent = nullptr )
        : QObject ( a_parent ) { }
    Class2* class2()
        { return &m_class2; }
private:
    Class2 m_class2;
};
#endif// CLASS1_H

Class3 & Class4: Aggregation

An instance of Class3 can exist without an instance of Class4. Example of composition is typically a Bicycle and its driver or passenger: without the Driver or Passenger it is still a Bicycle. It can function as a Bicycle.

Example of when you need to stop thinking about composition versus aggregation is whenever you say: without the other thing can in our software the first thing work.

Class3:

#ifndef CLASS3_H
#define CLASS3_H

#include <QObject>

#include <QPointer>
#include <Class4.h>

class Class3: public QObject
{
    Q_PROPERTY( Class4* class4 READ class4 WRITE setClass4 NOTIFY class4Changed)
public:
    Class3( QObject *a_parent = nullptr );
    Class4* class4() {
        return m_class4.data();
    }
    void setClass4 (Class4 *a_class4) {
         if ( m_class4 != a_class4 ) {
             m_class4 = a_class4;
             emit class4Changed();
         }
    }
signals:
    void class4Changed();
private:
    QPointer<Class4> m_class4;
};
#endif// CLASS3_H

Class5, Class6 & Class7: Shared composition
An instance of Class5 and-or an instance of Class6 can not exist without a instance of Class7 shared by Class5 and Class6. When one of Class5 or Class6 can and one can not exist without the shared instance, use QWeakPointer at that place.

Class5:

#ifndef CLASS5_H
#define CLASS5_H

#include <QObject>
#include <QSharedPointer>

#include <Class7.h>

class Class5: public QObject
{
    Q_PROPERTY( Class7* class7 READ class7 CONSTANT)
public:
    Class5( QObject *a_parent = nullptr, Class7 *a_class7 );
        : QObject ( a_parent )
        , m_class7 ( a_class7 ) { }
    Class7* class7()
        { return m_class7.data(); }
private:
    QSharedPointer<Class7> m_class7;
};

Class6:

#ifndef CLASS6_H
#define CLASS6_H

#include <QObject>
#include <QSharedPointer>

#include <Class7.h>

class Class6: public QObject
{
    Q_PROPERTY( Class7* class7 READ class7 CONSTANT)
public:
    Class6( QObject *a_parent = nullptr, Class7 *a_class7 )
        : QObject ( a_parent )
        , m_class7 ( a_class7 ) { }
    Class7* class7()
        { return m_class7.data(); }
private:
    QSharedPointer<Class7> m_class7;
};
#endif// CLASS6_H

Interfaces with QObject

FlyBehavior:

#ifndef FLYBEHAVIOR_H
#define FLYBEHAVIOR_H
#include <QObject>
// Don't inherit QObject here (you'll break multiple-implements)
class FlyBehavior {
    public:
        Q_INVOKABLE virtual void fly() = 0;
};
Q_DECLARE_INTERFACE(FlyBehavior , "be.codeminded.Flying.FlyBehavior /1.0") 
#endif// FLYBEHAVIOR_H

FlyWithWings:

#ifndef FLY_WITH_WINGS_H
#define FLY_WITH_WINGS_H
#include <QObject>  
#include <Flying/FlyBehavior.h>
// Do inherit QObject here (this is a concrete class)
class FlyWithWings: public QObject, public FlyBehavior
{
    Q_OBJECT
    Q_INTERFACES( FlyBehavior )
public:
    explicit FlyWithWings( QObject *a_parent = nullptr ): QObject ( *a_parent ) {}
    ~FlyWithWings() {}

    virtual void fly() Q_DECL_OVERRIDE;
}
#endif// FLY_WITH_WINGS_H

0 Add to favourites0 Bury

01 Jul 2016 11:38am GMT

22 Jun 2016

feedPlanet Maemo

2016-06-14 Meeting Minutes

Meeting held 2016-06-14 on FreeNode, channel #maemo-meeting (logs)

Attending:reinob, Win7Mac, M4rtinK, eekkelund, pichlo

Partial: chem|st

Absent: juiceme

Summary of topics (ordered by discussion):


(Topic Coding Competetion):

(Topic Bitcoins, HTTPS):


Action Items:
  • old items:
    • The next GA meeting should be announced soon.
    • Tax-exempt status
    • Could we make coding competetion happen
  • new items:
    • Find legal disclaimer for Bitcoin donations


Solved Action Items:
  • Ask about tor and https from techstaff

1 Add to favourites0 Bury

22 Jun 2016 7:27pm GMT

2016-06-07 Meeting Minutes

Meeting held 2016-06-07 on FreeNode, channel #maemo-meeting (logs)

Attending:reinob, eekkelund, Win7Mac, juiceme, pichlo

Partial:

Absent:

Summary of topics (ordered by discussion):


(Topic Tax-exempt):

(Topic More Coding Competetion discussion):


Action Items:
  • old items:
    • The next GA meeting should be announced soon.
    • Ask about tor and https from techstaff
    • Could we make coding competetion happen
  • new items:
    • Tax-exempt status


Solved Action Items:
  • Find out if https is doable (pichlo)
  • Check missing priviledges to the Council blog (juiceme)

0 Add to favourites0 Bury

22 Jun 2016 7:26pm GMT

07 Jun 2016

feedPlanet Maemo

2016-05-31 Meeting Minutes

Meeting held 2016-05-31 on FreeNode, channel #maemo-meeting (logs)

Attending: eekkelund, reinob, Win7Mac, juiceme

Partial: pichlo

Absent:

Summary of topics (ordered by discussion):


(Topic Coding competition):


Action Items:
  • old items:
    • The next GA meeting should be announced soon.
    • Ask about tor and https from techstaff (reinob)
  • new items:
    • Could we make coding competetion happen


Solved Action Items:
  • Check missing priviledges to the Council blog (juiceme)
  • Find out if https is doable (pichlo)
  • Arrange Council permissions on TMO to new Councillors. (juiceme)

0 Add to favourites0 Bury

07 Jun 2016 5:50pm GMT

2016-05-24 Meeting Minutes

Meeting held 2016-05-24 on FreeNode, channel #maemo-meeting (logs)

Attending: eekkelund, pichlo, Win7Mac, juiceme, reinob

Partial:

Absent:

Summary of topics (ordered by discussion):


(Topic https and tor support for maemo.org):

(Topic Missing priviligies for some councillors):


Action Items:
  • old items:
    • The next GA meeting should be announced soon.
  • new items:
    • Ask about tor and https from techstaff (reinob)
    • Check missing priviledges to the Council blog (juiceme)


Solved Action Items:
  • Find out if https is doable (pichlo)
  • Arrange Council permissions on TMO to new Councillors. (juiceme)
  • Add new Councillors to the mailing list and remove the old. (juiceme)
  • Get cloaks for new Councillors. (juiceme)

0 Add to favourites0 Bury

07 Jun 2016 5:48pm GMT

31 May 2016

feedPlanet Maemo

2016-05-17 Meeting Minutes

Meeting held 2016-05-17 on FreeNode, channel #maemo-meeting (logs)

Attending: eekkelund, pichlo, Win7Mac, juiceme, M4rtinK

Partial: chem|st

Absent: reinob

Summary of topics (ordered by discussion):


(Topic Tor users being blacklisted, read-only option for blacklisted IPs):

(Full httpS support for maemo.org):

(Spam to garage email):

(Bitcoin):


Action Items:
  • old items:
    • The next GA meeting should be announced soon.
    • Arrange Council permissions on TMO to new Councillors. (juiceme)
  • new items:
    • Find out if https is doable (pichlo)


Solved Action Items:
  • Add new Councillors to the mailing list and remove the old. (juiceme)
  • Get cloaks for new Councillors. (juiceme)

0 Add to favourites0 Bury

31 May 2016 7:04pm GMT

24 May 2016

feedPlanet Maemo

I/O bursts with QEMU 2.6

QEMU 2.6 was released a few days ago. One new feature that I have been working on is the new way to configure I/O limits in disk drives to allow bursts and increase the responsiveness of the virtual machine. In this post I'll try to explain how it works.

The basic settings

First I will summarize the basic settings that were already available in earlier versions of QEMU.

Two aspects of the disk I/O can be limited: the number of bytes per second and the number of operations per second (IOPS). For each one of them the user can set a global limit or separate limits for read and write operations. This gives us a total of six different parameters.

I/O limits can be set using the throttling.* parameters of -drive, or using the QMP block_set_io_throttle command. These are the names of the parameters for both cases:

-drive block_set_io_throttle
throttling.iops-total iops
throttling.iops-read iops_rd
throttling.iops-write iops_wr
throttling.bps-total bps
throttling.bps-read bps_rd
throttling.bps-write bps_wr

It is possible to set limits for both IOPS and bps at the same time, and for each case we can decide whether to have separate read and write limits or not, but if iops-total is set then neither iops-read nor iops-write can be set. The same applies to bps-total and bps-read/write.

The default value of these parameters is 0, and it means unlimited.

In its most basic usage, the user can add a drive to QEMU with a limit of, say, 100 IOPS with the following -drive line:

-drive file=hd0.qcow2,throttling.iops-total=100

We can do the same using QMP. In this case all these parameters are mandatory, so we must set to 0 the ones that we don't want to limit:

   { "execute": "block_set_io_throttle",
     "arguments": {
        "device": "virtio0",
        "iops": 100,
        "iops_rd": 0,
        "iops_wr": 0,
        "bps": 0,
        "bps_rd": 0,
        "bps_wr": 0
     }
   }

I/O bursts

While the settings that we have just seen are enough to prevent the virtual machine from performing too much I/O, it can be useful to allow the user to exceed those limits occasionally. This way we can have a more responsive VM that is able to cope better with peaks of activity while keeping the average limits lower the rest of the time.

Starting from QEMU 2.6, it is possible to allow the user to do bursts of I/O for a configurable amount of time. A burst is an amount of I/O that can exceed the basic limit, and there are two parameters that control them: their length and the maximum amount of I/O they allow. These two can be configured separately for each one of the six basic parameters described in the previous section, but here we'll use 'iops-total' as an example.

The I/O limit during bursts is set using 'iops-total-max', and the maximum length (in seconds) is set with 'iops-total-max-length'. So if we want to configure a drive with a basic limit of 100 IOPS and allow bursts of 2000 IOPS for 60 seconds, we would do it like this (the line is split for clarity):

   -drive file=hd0.qcow2,
          throttling.iops-total=100,
          throttling.iops-total-max=2000,
          throttling.iops-total-max-length=60

Or with QMP:

   { "execute": "block_set_io_throttle",
     "arguments": {
        "device": "virtio0",
        "iops": 100,
        "iops_rd": 0,
        "iops_wr": 0,
        "bps": 0,
        "bps_rd": 0,
        "bps_wr": 0,
        "iops_max": 2000,
        "iops_max_length": 60,
     }
   }

With this, the user can perform I/O on hd0.qcow2 at a rate of 2000 IOPS for 1 minute before it's throttled down to 100 IOPS.

The user will be able to do bursts again if there's a sufficiently long period of time with unused I/O (see below for details).

The default value for 'iops-total-max' is 0 and it means that bursts are not allowed. 'iops-total-max-length' can only be set if 'iops-total-max' is set as well, and its default value is 1 second.

Controlling the size of I/O operations

When applying IOPS limits all I/O operations are treated equally regardless of their size. This means that the user can take advantage of this in order to circumvent the limits and submit one huge I/O request instead of several smaller ones.

QEMU provides a setting called throttling.iops-size to prevent this from happening. This setting specifies the size (in bytes) of an I/O request for accounting purposes. Larger requests will be counted proportionally to this size.

For example, if iops-size is set to 4096 then an 8KB request will be counted as two, and a 6KB request will be counted as one and a half. This only applies to requests larger than iops-size: smaller requests will be always counted as one, no matter their size.

The default value of iops-size is 0 and it means that the size of the requests is never taken into account when applying IOPS limits.

Applying I/O limits to groups of disks

In all the examples so far we have seen how to apply limits to the I/O performed on individual drives, but QEMU allows grouping drives so they all share the same limits.

This feature is available since QEMU 2.4. Please refer to the post I wrote when it was published for more details.

The Leaky Bucket algorithm

I/O limits in QEMU are implemented using the leaky bucket algorithm (specifically the "Leaky bucket as a meter" variant).

This algorithm uses the analogy of a bucket that leaks water constantly. The water that gets into the bucket represents the I/O that has been performed, and no more I/O is allowed once the bucket is full.

To see the way this corresponds to the throttling parameters in QEMU, consider the following values:

  iops-total=100
  iops-total-max=2000
  iops-total-max-length=60

bucket

The bucket is initially empty, therefore water can be added until it's full at a rate of 2000 IOPS (the burst rate). Once the bucket is full we can only add as much water as it leaks, therefore the I/O rate is reduced to 100 IOPS. If we add less water than it leaks then the bucket will start to empty, allowing for bursts again.

Note that since water is leaking from the bucket even during bursts, it will take a bit more than 60 seconds at 2000 IOPS to fill it up. After those 60 seconds the bucket will have leaked 60 x 100 = 6000, allowing for 3 more seconds of I/O at 2000 IOPS.

Also, due to the way the algorithm works, longer burst can be done at a lower I/O rate, e.g. 1000 IOPS during 120 seconds.

Acknowledgments

As usual, my work in QEMU is sponsored by Outscale and has been made possible by Igalia and the help of the QEMU development team.

igalia-outscale

Enjoy QEMU 2.6!

0 Add to favourites0 Bury

24 May 2016 11:47am GMT