09 Nov 2011

feedPlanet OpenSolaris

Tim Foster: Self assembly – Part 2: multiple packages delivering configuration

Original image by bre pattis

Introduction

In the previous post in this series, we showed how it was possible to take a single package and republish it, such that other packages could overwrite a default configuration file.

The example we used was the Squid web proxy, allowing configuration packages to overwrite /etc/squid/squid.conf with new contents.

There was a limitation using that approach: only one package could contribute to that configuration at a time, replacing the entire shipped configuration.

Recall, that we define self-assembly in Chapter 1 of the IPS Developer Guide as:

Any collection of installed software on a system should be able to build itself into a working configuration when that system is booted, by the time the packaging operation completes, or at software runtime.

In this post, we'll cover a more advanced case than last time: true self-assembly, where the configuration can be delivered by multiple add-on packages, if necessary. In particular, we'll continue to talk about Squid, a package that isn't normally capable of self-assembly, and will show how we fix that.

How does self-assembly work?

The main premise with self-assembly, is that configuration for an application must be built from a composed view of all fragments of the entire configuration that are present on the system. That can be done either by the application itself, in which case nothing else is required on the part of the application packager, or it can be done with an add-on service to assemble the entire configuration file from the delivered fragments.

When a new package delivers another fragment of the configuration, then the application must have its configuration rebuilt to include that fragment.

Similarly, when a fragment is removed from the system, again, the application must have its configuration rebuilt from the remaining fragments on the system.

A good example of self-assembly is in the Solaris package for pkg:/web/server/apache-22. Solaris ships a default httpd.conf file that has an Include directive that references /etc/apache2/2.2/conf.d.

Packages can deliver a new file to that directory, and use a refresh_fmri actuator causing the system to automatically to refresh the Apache instance
either after a pkg install operation has completed, or after a
pkg remove operation has completed, causing the webserver to rebuild its configuration.

The reason behind self-assembly, is to replace postinstall, preinstall, preremove, postremove and class action scripts, needed by other packaging systems. Install-time scripting was a common source of errors during application packaging because the scripting had to work in multiple scenarios.

For example, scripts had to correctly run

With IPS, we eliminated those forms of install-time scripting, concentrating on an atomic set of actions (discussed in Chapter 3 of the IPS Developer Guide) that performed common packaging tasks, and allowing for actuators (discussed in Chapter 9 of the IPS Developer Guide) to run during packaging operations.

Actuators enable self-assembly to work on live systems by restarting or refreshing the necessary SMF services. Since the same SMF services they point to run during boot as well, we don't need to do anything when performing operations on alternate images: the next time the image is booted, our self-assembly is completed.

Making Squid self-assembly aware

As in the previous post, we will start by downloading and modifying our Squid package.

This time, we intend to remove the etc/squid/squid.conf file entirely - our self-assembly service will be constructing this file instead for us. Recall that
Squid delivers some of its configuration files with the following actions:

file 7d8f133b331e7460fbbdca593bff31446f8a3bad path=etc/squid/squid.conf \
    owner=root group=webservd mode=0644 \
    chash=272ed7f686ce409a121f427a5b0bf75aed0e2095 \
    original_name=SUNWsquid:etc/squid/squid.conf pkg.csize=1414 pkg.size=3409 \
    preserve=renamenew
file 7d8f133b331e7460fbbdca593bff31446f8a3bad \
    path=etc/squid/squid.conf.default owner=root group=bin mode=0444 \
    chash=272ed7f686ce409a121f427a5b0bf75aed0e2095 pkg.csize=1414 pkg.size=3409
file 971681745b21a3d88481dbadeea6ce7f87b0070a \
    path=etc/squid/squid.conf.documented owner=root group=bin mode=0444 \
    chash=b9662e497184c97fff50b1c249a6e153c51432e1 pkg.csize=60605 \
    pkg.size=200255

Since squid.conf.default is already shipped and is identical to the
squid.conf file that is also delivered, we can use that for the basis of our self-assembly of the squid.conf configuration file.

We download a copy of the package with the following command:

$ pkgrecv -s http://pkg.oracle.com/solaris/release --raw -d squid-proto squid@3.1.8,5.11-0.175.0.0.0.2.537

which pulls the content into the squid-proto directory.

We'll use a series of pkgmogrify(1) transforms to edit the package contents, similar to the ones we used in the previous post. We will remove the file action that delivers squid.conf using a drop transform operation, and will also deliver a new directory, etc/squid/conf.d. Here is the transform file that accomplishes that:

<transform set name=pkg.fmri -> edit value pkg://[^/]+/ pkg://mypublisher/>
<transform file path=etc/squid/squid.conf$ -> drop>
dir path=etc/squid/conf.d owner=root group=bin mode=0755

We can create a new manifest using this transform using pkgmogrify(1):

$ pkgmogrify squid\-assembly.mog \
    squid-proto/web%2Fproxy%2Fsquid/3.1.8%2C5.11-0.175.0.0.0.2.537%3A20111019T121425Z/manifest \
    > squid-assembly.mf

A self-assembly SMF service

In order for self-assembly to happen during packaging operations, we need to use an actuator discussed in Chapter 9 of the IPS Developer Guide.

The actuator is a special tag on any IPS action that points to an SMF service. The SMF service is made up of two components:

This self-assembly SMF service is going to be responsible for building the contents of /etc/squid/squid.conf. We'll talk about each component in the following section:

SMF manifest

This is what the SMF manifest of our self-assembly service looks like:

<?xml version="1.0"?>
<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">

<service_bundle type='manifest' name='Squid:self-assembly'>

<service
    name='config/network/http/squid-assembly'
    type='service'
    version='1'>

    <single_instance />

    <dependency
        name='fs-local'
        grouping='require_all'
        restart_on='none'
        type='service'>
            <service_fmri value='svc:/system/filesystem/local:default' />
    </dependency>

    <dependent
            name='squid_self-assembly-complete'
            grouping='optional_all'
            restart_on='none'>
            <service_fmri value='svc:/milestone/self-assembly-complete' />
    </dependent>
    <instance enabled='true' name='default'>
            <exec_method
                type='method'
                name='start'
                exec='/lib/svc/method/squid-self-assembly'
                timeout_seconds='30'/>

            <exec_method
                type='method'
                name='stop'
                exec=':true'
                timeout_seconds='0'/>

            <property_group name='startd' type='framework'>
                <propval name='duration' type='astring' value='transient' />
            </property_group>
    </instance>
</service>
</service_bundle>

This defines a service instance that we intend to use whenever we deliver new configuration file fragments to the system.

For that to happen, any configuration file fragment added or removed must include a restart_fmri actuator.

For example, a package might deliver a configuration file fragment:

file path=etc/squid/squid.conf/myconfig.conf owner=root group=bin mode=0644 \
    restart_fmri=svc:/config/network/http/squid-assembly:default \
    restart_fmri=svc:/network/http/squid:default

The other vital thing needed, is an SMF dependency on the SMF service delivered by the Squid package. We need to add this, so that the Squid application will only be able to start once our self-assembly service has finished producing our configuration file.

First, we'll create a proto area for the files we're going to add to our Squid package, and copy the default SMF manifest:

$ mkdir -p squid-assembly-proto/lib/svc/manifest/network
$ cp /lib/svc/manifest/network/http\-squid.xml squid-assembly-proto/lib/svc/manifest/network

Next, we edit the http-squid.xml SMF manifest, adding the following:

<!--
  Wait for the Squid self-assembly service to complete
-->
<dependency name='squid-assembly'
    grouping='require_all'
    restart_on='none'
    type='service'>
    <service_fmri
        value='svc:/config/network/http/squid-assembly:default'/>
</dependency>

Now that we've done this, our next step, is writing the method script for our self-assembly service.

The SMF method script

We need to write a script, such that when it is run, we end up with /etc/squid.conf containing all changes, as defined in all configuration fragments installed on the system.

This step can be as simple or complex as you'd like it to be - essentially we're performing postinstall scripting here, but on our terms: we know exactly the environment the script is running in - that of a booted OS where our package is installed (defined by the depend actions that accompany the package)

Here is a sample script, written in Python (as short as I could make it, so there's very little error checking involved here) which takes squid.conf.default copies it to squid.conf, then applies a series of edits to it.

We'll save the script as /lib/svc/method/squid-self-assembly.

#!/usr/bin/python2.6
import shutil
import os
import re
import logging

# define the paths we'll work with
MASTER="/etc/squid/squid.conf.default"
CONF_FILE="/etc/squid/squid.conf"
CONF_DIR="/etc/squid/conf.d/"

# verbose logging for now
logging.basicConfig(level=logging.DEBUG)

def apply_edits(fragment):
        """Takes edit operations in the path "fragment", and applies
        them to CONF_FILE in order. The syntax of our config file is
        intentionally basic. We support the following operations:

        # lines that start with a hash are comments
        add <line to add to the config file>
        remove <regular expression to remove>
        """

        squid_config = file(CONF_FILE).readlines()
        squid_config = "".join(squid_config)

        # read our list of operations
        operations = open(fragment, "r").readlines()
        operations = [line.rstrip() for line in operations]
        for op in operations:
                if op.startswith("add"):
                        addition = op[len("add") + 1:]
                        logging.debug("adding line %s" % addition)
                        squid_config += "\n" + addition
                elif op.startswith("remove"):
                        exp = op[len("remove") + 1:]
                        squid_config = re.sub(exp, "", squid_config)
                        logging.debug("removing expression %s" % exp)
                elif op.startswith("#"):
                        pass

        conf = open(CONF_FILE, "w")
        conf.write(squid_config + "\n")
        conf.close()

# first, remove any existing configuration
if os.path.exists(CONF_FILE):
       os.unlink(CONF_FILE)

# now copy the master template file in, on
# which all edits are based
shutil.copy(MASTER, CONF_FILE)
os.chmod(CONF_FILE, 0644)

fragments = []
# now iterate through the contents of /etc/squid/conf.d
# looking for configuration fragments, and apply the changes
# find in a defined order.   We do not look in subdirectories.
for dirpath, dirnames, filenames in os.walk("/etc/squid/conf.d/"):
        fragments = sorted(filenames)
        break

for fragment in fragments:
        logging.debug("Applying edits from %s" % fragment)
        apply_edits(os.path.join(CONF_DIR, fragment))

Testing the self-assembly script

We can now test the self-assembly script. For the most part, this testing can be done outside the confines of the pkg(1) command - we simply need to ensure
that our self-assembly script runs properly.

First, we'll check that the squid.conf file isn't present, run the script, then determine that the contents are the same as squid.conf.default

# ls /etc/squid/squid.conf
/etc/squid/squid.conf: No such file or directory
# /lib/svc/method/squid-self-assembly
# digest -a sha1 /etc/squid/squid.conf.default /etc/squid/squid.conf
(/etc/squid/squid.conf.default) = 7d8f133b331e7460fbbdca593bff31446f8a3bad
(/etc/squid/squid.conf) = 7d8f133b331e7460fbbdca593bff31446f8a3bad
#

Next, we'll try a simple configuration fragment:

# cat > /etc/squid/conf.d/change_http_port.conf
# The default configuration uses port 3128, our organisation uses 8080
# We'll remove that default, add a comment, and add a http_port directive
remove # Squid normally listens to port 3128
remove http_port 3128
add # Our organisation requires Squid to operate on port 8080
add http_port 8080
^D

Then we'll test the self-assembly script again:

# /lib/svc/method/squid-self-assembly
DEBUG:root:  --- applying edits from change_http_port.conf   ---
DEBUG:root:removing expression # Squid normally listens to port 3128
DEBUG:root:removing expression http_port 3128
DEBUG:root:adding line # Our organisation requires Squid to operate on port 8080
DEBUG:root:adding line http_port 8080

We can verify that the changes have been made:

# grep "port 8080" /etc/squid/squid.conf
# Our organisation requires Squid to operate on port 8080
http_port 8080

Now, we'll add another configuration fragment:

# cat > /etc/squid/conf.d/connect_ports.conf
# We want to allow users to connect to gmail and irc
# over our proxy server.
add # We need to allow access to gmail and irc
add acl Connect_ports port 5222     # gmail chat
add acl Connect_ports port 6667     # irc chat
add http_access allow CONNECT Connect_ports
^D

and see what happens when we run the self-assembly script:

# /lib/svc/method/squid-self-assembly
DEBUG:root:  --- applying edits from change_http_port.conf   ---
DEBUG:root:removing expression # Squid normally listens to port 3128
DEBUG:root:removing expression http_port 3128
DEBUG:root:adding line # Our organisation requires Squid to operate on port 8080
DEBUG:root:adding line http_port 8080
DEBUG:root:  --- applying edits from connect_ports.conf   ---
DEBUG:root:adding line # We need to allow access to gmail and irc
DEBUG:root:adding line acl Connect_ports port 5222     # gmail chat
DEBUG:root:adding line acl Connect_ports port 6667     # irc chat
DEBUG:root:adding line http_access allow CONNECT Connect_ports

Again, we can verify that the edits have been made correctly:

# grep "port 8080" /etc/squid/squid.conf
# Our organisation requires Squid to operate on port 8080
http_port 8080
# egrep gmail\|irc /etc/squid/squid.conf
# We need to allow access to gmail and irc
acl Connect_ports port 5222     # gmail chat
acl Connect_ports port 6667     # irc chat

And finally, we can see what happens if we remove one of our fragments:

# rm /etc/squid/conf.d/connect_ports.conf
# /lib/svc/method/squid-self-assembly
DEBUG:root:  --- applying edits from change_http_port.conf   ---
DEBUG:root:removing expression # Squid normally listens to port 3128
DEBUG:root:removing expression http_port 3128
DEBUG:root:adding line # Our organisation requires Squid to operate on port 8080
DEBUG:root:adding line http_port 8080
#
# grep "port 8080" /etc/squid/squid.conf
# Our organisation requires Squid to operate on port 8080
http_port 8080
# egrep gmail\|irc /etc/squid/squid.conf
#

As expected, the configuration file no longer contains the directives configured by connect_ports.conf, since that was removed from the system, but still
contains the changes from change_http_port.conf

Delivering the SMF service

The bulk of the hard work has been done now - to recap:

All that remains, is to ensure that the self-assembly service gets included in
the Squid package.

For that, we'll add a few more lines to the pkgmogrify(1) transform that we talked about earlier, so that it looks like:

<transform set name=pkg.fmri -> edit value pkg://[^/]+/ pkg://mypublisher/>
<transform file path=etc/squid/squid.conf$ -> drop>
dir path=etc/squid/conf.d owner=root group=bin mode=0755
file path=lib/svc/method/squid/squid-self-assembly group=bin mode=0555 owner=root
file path=lib/svc/manifest/network/http-squid-assembly.xml group=sys \
    mode=0444 owner=root restart_fmri=svc:/system/manifest-import:default

Now we can transform our original Squid package, and publish it to our repository:

$ pkgmogrify squid-assembly.mog \
    squid-proto/web%2Fproxy%2Fsquid/3.1.8%2C5.11-0.175.0.0.0.2.537%3A20111019T121425Z/manifest \
    > squid\-assembly.mf
$ pkgsend -s myrepository publish -d squid-assembly-proto \
    -d squid-proto/web%2Fproxy%2Fsquid/3.1.8%2C5.11-0.175.0.0.0.2.537%3A20111019T121425Z \
    squid\-assembly.mf
WARNING: Omitting signature action 'signature 2ce2688faa049abe9d5dceeeabc4b17e7b72e792
.
.
pkg://mypublisher/web/proxy/squid@3.1.8,5.11-0.175.0.0.0.2.537:20111108T201820Z
PUBLISHED
$

Installing that package, we discover a svc:/config/network/http/squid-assembly service, and verify that when we drop unpackaged files into /etc/squid/conf.d, and restart the self-assembly service, we see what we expect:

# more /var/svc/log/config-network-http-squid-assembly\:default.log
[ Nov  8 12:19:50 Enabled. ]
[ Nov  8 12:19:50 Rereading configuration. ]
[ Nov  8 12:19:50 Executing start method ("/lib/svc/method/squid-self-assembly"). ]
[ Nov  8 12:19:50 Method "start" exited with status 0. ]
[ Nov  8 12:23:42 Stopping because service restarting. ]
[ Nov  8 12:23:42 Executing stop method (null). ]
[ Nov  8 12:23:42 Executing start method ("/lib/svc/method/squid-self-assembly"). ]
DEBUG:root:  --- applying edits from change_port.conf   ---
DEBUG:root:removing expression # Squid normally listens to port 3128
DEBUG:root:removing expression http_port 3128
DEBUG:root:adding line # Our organisation requires Squid to operate on port 8080
DEBUG:root:adding line http_port 8080
[ Nov  8 12:23:42 Method "start" exited with status 0. ]

We have verified that Squid is performing self-assembly perfectly.

Delivering new configuration fragments

Now that we have a service that's capable of performing self-assembly, we need to know how to deliver configuration fragments in new packages.

This is simply a case of delivering config files to /etc/squid/conf.d, and applying the correct actuator tags to the manifest.

An example manifest that delivers this would be:

set name=pkg.fmri value=pkg:/config/web/proxy/squid-configuration@2.0
set name=pkg.summary value="Our organisations squid configurations"
file path=etc/squid/conf.d/change_http.conf owner=root group=bin mode=0644 \
    restart_fmri=svc:/config/network/http/squid-assembly:default \
    restart_fmri=svc:/network/http:squid

When we publish, then install this manifest, we see:

# pkg install squid-configuration@2
           Packages to install:  1
       Create boot environment: No
Create backup boot environment: No
            Services to change:  2

DOWNLOAD                                  PKGS       FILES    XFER (MB)
Completed                                  1/1         1/1      0.0/0.0

PHASE                                        ACTIONS
Install Phase                                    3/3

PHASE                                          ITEMS
Package State Update Phase                       1/1
Image State Update Phase                         2/2

We can quickly verify that the Squid configuration has changed:

$ curl localhost:8080 | grep squid/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2904  100  2904    0     0  1633k      0 --:--:-- --:--:-- --:--:-- 2835k
<p>Generated Tue, 08 Nov 2011 23:00:27 GMT by tcx2250-13 (squid/3.1.8)</p>

And we can backout the configuration by removing the package, and again check that the Squid configuration has changed:

# curl localhost:8080 | grep squid
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (7) couldn't connect to host
# curl localhost:3128 | grep squid/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3140  100  3140    0     0  1779k      0 --:--:-- --:--:-- --:--:-- 3066k
<p>Generated Tue, 08 Nov 2011 23:03:37 GMT by tcx2250-13 (squid/3.1.8)</p>

We won't go into details here, but clearly, multiple packages could deliver
configuration fragments at the same time, and they would all contribute to the
configuration of our service.

Conclusion

This has been a pretty fast example of the self-assembly idiom, but we hope this has been useful, and shows complex scripting operations can be performed in IPS.

There may more work to do to make the Squid application fully self-assembly aware - we've only covered the main configuration file and have't looked at whether we also want to allow the other files in /etc/squid to participate in self-assembly. If we did want to do that, it would be a case of ensuring that:

Of course, there's other ways in which a self-assembly service could perform edits - we could use SMF to deliver properties to the service, which are then accessed by a self-assembly script, and placed into a configuration file, but perhaps that's an example for another day.


Filed under: IPS, OpenSolaris

09 Nov 2011 6:05pm GMT

Tim Foster: IPS Self-assembly – Part 1: overlays

Original image by 3liz4

Introduction

I'm starting a small series of blog posts to talk about one of the important concepts in IPS - self-assembly. We cover this in the IPS Developer Guide but don't provide many examples as yet.

In the IPS Developer Guide, we introduced the concept of self-assembly as:

Any collection of installed software on a system should be able to build itself into a working configuration when that system is booted, by the time the packaging operation completes, or at software runtime.

Lots of software ships with default configuration in sample files, often installed in /etc. During packaging, these files are commonly marked as "user editable", with an attribute defining how those user edits should be treated in the case where the shipped example file gets updated in new release of the package.

In IPS, those user editable files are marked with a preserve attribute, which is documented in the pkg(5) man page.

However, what happens if we want to allow another package to deliver new configuration instead of simply allowing user edits?

By default, IPS will report an error if two packages try to deliver the same file.

In these blog posts, we'll take a sample package, and show how it can be modified to allow us to deliver new add-on packages that deliver different configuration.

Before getting into a more complicated true self-assembly scenario (in the next post), we'll cover a very simple one first.

In this first post, we'll talk about the overlay attribute. Technically, this example doesn't actually cover self-assembly. Instead, it shows how IPS allows packages to re-deliver configuration files already delivered by another package.

First, let's introduce our example package.

Our example package

We'll use a package that already exists as our example: the Squid web proxy.

In our examples, we're going to delivering a new version of Squid that allows us to achieve our goal of being able to deliver add-on packages to supply configuration.

To be clear, I'm not suggesting all administrators ought to do this - by using their own private copy of a package shipped by Oracle, they face the burden of maintaining this version themselves: future upgrades from the solaris publisher will not automatically update their version. By default, publishers in IPS are sticky - so packages installed from one publisher may not be updated by a new version of that package from another publisher.

Publisher stickiness may be overridden, but then the administrator risks that their carefully crafted package gets updated by a version of the package from Oracle. In addition, the presence of a local version of the package may also prevent updates from occurring.

However, when I was looking for an example of the modifications that need to be made to a package which doesn't normally participate in self-assembly, Squid fits the bill nicely.

Let's look at the choices that were made when Squid was being packaged for Solaris, concentrating on how its configuration files are handled.

Using the following command, we can show the actions associated with the squid.conf files that are delivered in the package:

$ pkg contents -H -r -o action.raw -a path=etc/squid/squid.conf* squid | pkgfmt

Here is the output from the command:

file 7d8f133b331e7460fbbdca593bff31446f8a3bad path=etc/squid/squid.conf \
    owner=root group=webservd mode=0644 \
    chash=272ed7f686ce409a121f427a5b0bf75aed0e2095 \
    original_name=SUNWsquid:etc/squid/squid.conf pkg.csize=1414 pkg.size=3409 \
    preserve=renamenew
file 7d8f133b331e7460fbbdca593bff31446f8a3bad \
    path=etc/squid/squid.conf.default owner=root group=bin mode=0444 \
    chash=272ed7f686ce409a121f427a5b0bf75aed0e2095 pkg.csize=1414 pkg.size=3409
file 971681745b21a3d88481dbadeea6ce7f87b0070a \
    path=etc/squid/squid.conf.documented owner=root group=bin mode=0444 \
    chash=b9662e497184c97fff50b1c249a6e153c51432e1 pkg.csize=60605 \
    pkg.size=200255

We can see that the package delivers three files:

etc/squid/squid.conf
This is the default configuration file that squid uses. You can see that it has a preserve attribute, with a value set to renamenew User edits to this file are allowed, and will be preserved on upgrade, and any new versions of the file (delivered by an updated Squid package) will be renamed.
etc/squid/squid.conf.default
Squid also ships with a second copy of the configuration file (notice how the hashes are the same as the previous version) with a different name - presumably to use as a record of the original configuration.
etc/squid/squid.conf.documented
Finally we have another copy of the configuration file, this time with more comments included, to better explain the configuration.

Adding an overlay attribute

In IPS, two packages are allowed to deliver the same file if:

In both cases, all other file attributes (owner, mode, group) must match. The overlay attribute is covered in Chapter 3 of the IPS Developer Guide and is also documented in the pkg(5) man page.

Since our sample package doesn't deliver its configuration file, etc/squid/squid.conf, with an overlay attribute, we'll need to modify the package.

First, we download the package in a raw form, suitable for republishing later, and show where pkgrecv(1) stores the manifest:

$ pkgrecv -s http://pkg.oracle.com/solaris/release --raw -d squid-proto squid@3.1.8,5.11-0.175.0.0.0.2.537
Processing packages for publisher solaris ...
Retrieving and evaluating 1 package(s)...
PROCESS                                         ITEMS     GET (MB)    SEND (MB)
Completed                                         1/1    18.0/18.0      0.0/0.0

$ find squid-proto -name manifest
squid-proto/web%2Fproxy%2Fsquid/3.1.8%2C5.11-0.175.0.0.0.2.537%3A20111019T121425Z/manifest

Next, we'll define a simple pkgmogrify(1) transform to add an overlay=allow attribute.

We'll also remove the solaris publisher from the FMRI, as we intend to republish this package to our own repository. (This transform is discussed in more detail in Chapter 14 of the IPS Developer Guide)

The transform looks like:

<transform set name=pkg.fmri -> edit value pkg://[^/]+/ pkg://mypublisher/>
<transform file path=etc/squid/squid.conf$ -> set overlay allow>

Here's how we run it:

$ pkgmogrify squid-overlay.mog \
    squid-proto/web%2Fproxy%2Fsquid/3.1.8%2C5.11-0.175.0.0.0.2.537%3A20111019T121425Z/manifest \
    > squid-overlay.mf

Finally we can republish our package:

$ pkgsend -s myrepository publish \
    -d squid-proto/web%2Fproxy%2Fsquid/3.1.8%2C5.11-0.175.0.0.0.2.537%3A20111019T121425Z \
    squid-overlay.mf
WARNING: Omitting signature action 'signature 2ce2688faa049abe9d5dceeeabc4b17e7b72e792
.
.
.
pkg://mypublisher/web/proxy/squid@3.1.8,5.11-0.175.0.0.0.2.537:20111108T220909Z
PUBLISHED

We get a warning when republishing it saying that we're dropping the signature action (I've trimmed the output here).

Package signing is always performed on a repository using pkgsign(1), never on a manifest. Since the package's timestamp is always updated on publication, that would cause any hardcoded signatures to be invalid. Package signing is covered in more detail in Chapter 11 of the IPS Developer Guide.

This gets us part of the way towards our goal: we've now got a version of Squid that can allow other packages to deliver a new copy of etc/squid/squid.conf.

Notice that we've left the version alone on our copy of Squid, so it still complies with the same package version constraints that were on the original version of Squid that was shipped with Solaris.

Writing Configuration Packages

At this point, we can start writing packages to deliver new versions of our configuration file.

First let's install our modified squid package. We'll add our local repository to the system, and make sure we search for packages there before the solaris publisher, so that our packages are discovered first.

$ pfexec pkg set-publisher --search-before=solaris -p ./myrepository
Updated publisher(s): mypublisher
$ pfexec pkg install squid
           Packages to install:  1
       Create boot environment: No
Create backup boot environment: No
            Services to change:  1

DOWNLOAD                                  PKGS       FILES    XFER (MB)
Completed                                  1/1   1519/1519      8.5/8.5

PHASE                                        ACTIONS
Install Phase                              1704/1704

PHASE                                          ITEMS
Package State Update Phase                       1/1
Image State Update Phase                         2/2

Next, we'll create our configuration package. Perhaps the only thing we want to change, is the default port that Squid listens on. Let's write a new squid.conf file that uses port 8080 instead of 3128:

Our original squid configuration shows:

$ grep 3128 /etc/squid/squid.conf
# Squid normally listens to port 3128
http_port 3128

We'll write our new configuration:

$ mkdir -p squid-conf-proto/etc/squid
$ cat /etc/squid/squid.conf | sed -e 's/3128/8080/g' \
    > squid-conf-proto/etc/squid/squid.conf
$ grep 8080 squid-conf-proto/etc/squid/squid.conf
# Squid normally listens to port 8080
http_port 8080

Now, we'll create a package for the file. We'll make the package depend on our Squid package. For this package, since the Squid package already delivers the dir action needed for etc/squid we'll just deliver the file-action for our new squid.conf.

$ cat > squid-conf.mf
set name=pkg.fmri value=config/web/proxy/squid-configuration@1.0
set name=pkg.summary value="My Company Inc. Default squid.conf settings"
file path=etc/squid/squid.conf owner=root group=webservd mode=0644 \
    overlay=true preserve=renameold
depend type=require fmri=web/proxy/squid@3.1.8
^D

Notice that we have specified overlay=true to indicate that this action should overlay any existing file, and have specified preserve=renameold to indicate that we want the old file renamed if one exists.

$ pkgsend -s myrepository publish -d squid-conf-proto squid-conf.mf
pkg://mypublisher/config/web/proxy/squid-configuration@1.0,5.11:20111108T150325Z
PUBLISHED

We can now install this package to our system, and check to make sure our changes have appeared:

$ pfexec pkg install squid-configuration
           Packages to install:  1
       Create boot environment: No
Create backup boot environment: No

DOWNLOAD                                  PKGS       FILES    XFER (MB)
Completed                                  1/1         1/1      0.0/0.0

PHASE                                        ACTIONS
Install Phase                                    4/4

PHASE                                          ITEMS
Package State Update Phase                       1/1
Image State Update Phase                         2/2

The following unexpected or editable files and directories were
salvaged while executing the requested package operation; they
have been moved to the displayed location in the image:

  etc/squid/squid.conf -> /var/pkg/lost+found/etc/squid/squid.conf-20111108T071810Z

$ grep 8080 /etc/squid/*
/etc/squid/squid.conf:# Squid normally listens to port 8080
/etc/squid/squid.conf:http_port 8080
$ pkg list squid squid-configuration
NAME (PUBLISHER)                                  VERSION                    IFO
config/web/proxy/squid-configuration              1.0                        i--
web/proxy/squid                                   3.1.8-0.175.0.0.0.2.537    i--

Conclusion

This was a pretty simple case - we've simply modified an existing package, and delivered a single new package allowing a single configuration package to deliver a change to the file.

This wasn't really self-assembly per se, since the configuration is still hard-coded, but it is a common use-case, and provides a good introduction to our next example.

However, what happens if we want to deliver a further change to this file, from another package? Trying the same approach again, creating a new package "pkg:/config/web/proxy/squid-configuration-redux" then trying to install it,
we see:

$ pkgsend -s myrepository publish -d squid-conf-proto squid-conf-redux.mf
pkg://mypublisher/config/web/proxy/squid-configuration-redux@1.0,5.11:20111108T152449Z
PUBLISHED

$ pfexec pkg install squid-configuration-redux
Creating Plan |
pkg install: The following packages all deliver file actions to etc/squid/squid.conf:

  pkg://mypublisher/web/proxy/squid@3.1.8,5.11-0.175.0.0.0.2.537:20111108T151647Z
  pkg://mypublisher/config/web/proxy/squid-configuration-redux@1.0,5.11:20111108T152449Z
  pkg://mypublisher/config/web/proxy/squid-configuration@1.0,5.11:20111108T151420Z

These packages may not be installed together. Any non-conflicting set may
be, or the packages must be corrected before they can be installed.

So IPS only allows one configuration package to be installed at a time. We'll uninstall our configuration package, revert the old squid.conf content, then install our new configuration package:

$ pfexec pkg uninstall squid-configuration
            Packages to remove:  1
       Create boot environment: No
Create backup boot environment: No

PHASE                                        ACTIONS
Removal Phase                                    3/3

PHASE                                          ITEMS
Package State Update Phase                       1/1
Package Cache Update Phase                       1/1
Image State Update Phase                         2/2

$ pfexec pkg revert /etc/squid/squid.conf
            Packages to update:  1
       Create boot environment: No
Create backup boot environment: No

DOWNLOAD                                  PKGS       FILES    XFER (MB)
Completed                                  1/1         1/1      0.0/0.0

PHASE                                        ACTIONS
Update Phase                                     1/1

PHASE                                          ITEMS
Image State Update Phase                         2/2
$ pfexec pkg install squid-configuration-redux
           Packages to install:  1
       Create boot environment: No
Create backup boot environment: No

DOWNLOAD                                  PKGS       FILES    XFER (MB)
Completed                                  1/1         1/1      0.0/0.0

PHASE                                        ACTIONS
Install Phase                                    4/4

PHASE                                          ITEMS
Package State Update Phase                       1/1
Image State Update Phase                         2/2

The following unexpected or editable files and directories were
salvaged while executing the requested package operation; they
have been moved to the displayed location in the image:

  etc/squid/squid.conf -> /var/pkg/lost+found/etc/squid/squid.conf-20111108T072930Z

We see that the new configuration file has been installed.

In the next post in this series, we'll provide a more complex example of
self-assembly.


Filed under: IPS, OpenSolaris

09 Nov 2011 6:00pm GMT

Tim Foster: The IPS System Repository

Original image by nori_n

I'm excited about today's launch of Solaris 11 - I've been contributing to Solaris for quite a while now, pretty much since 1996, but my involvement in S11 has been the most fun I've had in all releases so far.

I've talked before about some of the work I've done on IPS over the last two years - pkg history, pkgdepend (and here), pkglint and pkgsend and most recently, helping to put together the IPS Developer Guide.

Today, I'm going to talk about the system repository and how I helped.

How zones differ from earlier releases

Zones that use IPS are different than those in Solaris 10, in that they are always full-root: every zone contains its own local copy of each package, they don't inherit packaged content from the global zone as "sparse" zones did in Solaris 10.

This simplifies a lot of zone-related functionality: for the most part, administrators can treat a zone as if it were a full Solaris instance, albeit a very small one. By default new zones in S11 are tiny. However, packaging with zones is a little more complex, and the system aims to hide that complexity
from users.

Some packages in the zone always need to be kept in sync with those packages in the global zone. For example, anything which delivers a kernel module and a userland application that interfaces with it must be kept in sync between the global zone and any non-global zones on the system.

In earlier OpenSolaris releases, after each global-zone update, each non-global zone had to be updated by hand, attaching and detaching each zone. During that detach/attach the ipkg brand scripts determined which packages were now in the global zone, and updated the non-global zone accordingly.

In addition, in OpenSolaris, the packaging system itself didn't have any way of ensuring that every publisher in the global zone was also available in the non-global zone, making updates difficult if switching publishers.

Zones in Solaris 11

In Solaris 11, zones are now first-class citizens of the packaging system. Each zone is installed as a linked image, connected to the parent image, which is the global zone.

During packaging operations in the global zone, IPS recurses into any non-global zones to ensure that packages which need to be kept in sync between the global and non-global zones are kept in sync.

For this to happen, it's important for the zone to have access to all of the IPS repositories that are available from the global zone.

This is problematic for a few reasons:

The System Repository

The system repository, and accompanying zones-proxy services was our solution to the list of problems above.

The SMF Services responsible are:

The first two services run in the global zone, the last one runs in the non-global zones.

With these services, the system repository shares publisher configuration to all non-global zones on the system, and also acts as a conduit to the publishers configured in the global zone. Inside the non-global zone, these proxied global-zone publishers are called system publishers.

When performing packaging operations inside a zone that accesses those publishers, Solaris proxies access through the system repository. While proxying, the system repository also caches any file-content that was
downloaded. If there are lots of zones all downloading the same packaged content, that will be efficiently managed.

Implementation

If you don't care about how all this works behind the scenes, then you can stop reading now.

There's three parts to making all of the above work, apart from the initial linked image functionality that Ed worked on, which was fundamental to all of the system repository work.

IPS client/repository support

Brock managed the heavy lifting here. This work involved:

Zones proxy

The zones proxy client, when started in the non-global zone creates a socket which listens on an inet port on 127.0.0.1. It passes the file descriptor for this socket to the zones proxy daemon via a door call.

The zones proxy daemon then listens for connections on the file descriptor. When the zone proxy daemon receives a connection, it proxies the connection to the system repository.

This allows the zone to access the system repository without any additional networking configuration needed (which I think is pretty neat - nicely done Krister!)

System repository

The system repository itself consists of two components:

Brock initially prototyped some httpd.conf configurations, and I worked on the code to write them automatically, produce the response that the system repository would use to inform zones of the configured publishers, and also worked out how to proxy access to file-based publishers in the global zone, which was an interesting problem to solve.

When you start the system-repository service in the global zone, pkg.sysrepo(1) determines the enabled, configured publishers then creates a response file served to non-global zones that want to discover the publishers configured in the global zone. It then uses a Mako template from /etc/pkg/sysrepo/sysrepo_httpd.conf.mako to generate an Apache configuration file.

The configuration file describes a basic caching proxy, providing limited access to the URLs of each publisher, as well as allowing URL rewrites to serve any file-based repositories. It uses the SSL keys and certificates from the global zone, and allows proxies access to those from the non-global zone over http.
(remember, data served by the system repository between the zone and non-global zone goes over the zones proxy socket, so http is fine here: access from the proxy to the publisher still goes over https)

The system repository service then starts an Apache instance, and a daemon to keep the proxy cache down to its configured maximum size. More detail on the options available to tune the system repository are in pkg.sysrepo(1) man page.

Result?

The practical upshot of all this, is that all zones can access all publishers configured on the global zone, and if that configuration changes, the zones publishers automatically change too. Of course, non-global zones can add their own publishers, but aren't allowed to change the order, or disable any system
publishers.

Here's what the pkg publisher output looks like in a non-global zone:

root@puroto:~# pkg publisher
PUBLISHER                             TYPE     STATUS   URI
solaris                  (non-sticky, syspub) origin   online   proxy://http://pkg.oracle.com/solaris11/release/
mypublisher              (syspub)     origin   online   http://localhost:1008/mypublisher/89227627f3c003d11b1e4c0b5356a965ef7c9712/
test                     (syspub)     origin   online   http://localhost:1008/test/eec48b7c8b107bb3ec9b9cf0f119eb3d90b5303e/

and here's the system repository running in the global zone:

$ ps -fu pkg5srv | grep httpd
 pkg5srv   206  2334   0 12:02:02 ?           0:00 /usr/apache2/2.2/bin/64/httpd.worker -f /system/volatile/pkg/sysrepo/sysrepo_ht
 pkg5srv   204  2334   0 12:02:02 ?           0:00 /usr/apache2/2.2/bin/64/httpd.worker -f /system/volatile/pkg/sysrepo/sysrepo_ht
 pkg5srv   205  2334   0 12:02:02 ?           0:00 /usr/apache2/2.2/bin/64/httpd.worker -f /system/volatile/pkg/sysrepo/sysrepo_ht
 pkg5srv   939  2334   0 12:46:32 ?           0:00 /usr/apache2/2.2/bin/64/httpd.worker -f /system/volatile/pkg/sysrepo/sysrepo_ht

Personally, I've found this capability to be incredibly useful. I work from home, and have a system with an internet-facing non-global zone, and a global zone accessing our corporate VPN. My non-global zone is able to securely access new packages when it needs to (and I get to test my own code at the same time!)

Performing a pkg update from the global zone ensures that all zones are kept in sync, and will update all zones automatically (though, as mentioned in the Zones administration guide, pkg update <list of packages> will simply update the global zone, and ensure that during that update only the packages that cross the kernel/userland boundary are updated in each zone.)

Working on zones and the system repository was a lot of fun - hope you find it useful.


Filed under: IPS, OpenSolaris

09 Nov 2011 5:46pm GMT

Dave Stewart: THE YOCTO PROJECT AT THE 2011 EMBEDDED LINUX CONFERENCE - EUROPE

Cross-posting this from my blog on the Yocto Project site. Follow up with your comment here or on that other site.

Much has been written about how the Internet has revolutionized collaboration and made it possible for your brilliant ideas to make a difference no matter where you live on the planet. Bill Gates is famously quoted in Nick Kristoff's "The World is Flat" that "... so many people can plug and play from anywhere, natural talent has started to trump geography." This is of course true, but even with the Internet, there is no replacement for face-to-face interaction. The tribe, it seems, still needs to gather around the fire to have a talk now and then.
The Yocto Project booth at Embedded Linux Conf EuropeLinuxCon Europe / ELCE 2011

The conference setup was in a very new and modern hotel called the Hotel Clarion Congress in Prague, which was a terrific venue. Since the Yocto Project was a co-sponsor of the event, we got a nice booth location, and Tracey Erway did a fantastic job setting up the booth and populating it with demos, videos, giveaways and t-shirts. You can also see Darren Hart manning the booth behind the Gource video that he rendered showing the many contributors on the project from many places.

I was impressed by how much the booth became a gathering spot for people wanting to talk about Yocto and what we were doing in the project.
Dinner with Yocto team members and families, Prague

There was a lot of opportunity for people to interact with Richard Purdie, who is the Yocto Project architect, and a very approachable guy. Koen Kooi is a TI guy and a long-time Open Embedded and Angstrom maintainer, who really helped us out a lot in the booth, and showed off his Beagle Board as a demo.
LinuxCon Europe / ELCE 2011

I also appreciated the work of Jeff Osier-Mixon, better known as jefro, who is the Yocto Project Community Manager, and always helps us make sure we are taking care of the community and helping it to be nurtured and grow.
LinuxCon Europe / ELCE 2011

It was fun catching up with Marcin, who works for Linaro, but is a long time heavy contributor to Poky and OpenEmbedded. Here is a photo of him with Richard and Dirk Hohndel from Intel.
IMG_0386

And like any good Linux Foundation event, there were some excellent parties to give us a space to hang out with each other and appreciate the unique culture and food of Prague.
IMG_0367

But in spite of being in such a beautiful city, these folks are sometimes hard to break away from hacking. On a Saturday with nothing planned but some tourist activities, I actually had to "encourage" Saul to take a break from his computer. (I actually closed the lid of his laptop to make sure he actually stopped working. I hope he forgives me.)

Dinner with Yocto team members and families, Prague
Dinner with Yocto team members and families, Prague

More kudos to Sean Hudson (Mentor Graphics), Paul Eggleton (Intel), Bill Mills (TI), Nithya Ruff (Wind River), Philip Ballister (OpenSDR), Jessica Zhang (Intel) and too many others to count who helped us so much in the talks, booth and discussions about Yocto.

Oh yeah, I guess there were some more or less official things going on as well:

The problem with mentioning anyone in a blog post like this is that I'm sure I have missed somebody who will be hurt because I didn't mention them. I am so sorry about that, and I hope you can forgivde me!

09 Nov 2011 5:34pm GMT

07 Nov 2011

feedPlanet OpenSolaris

Peter Tribble: Zooming into images

I spend rather a lot of my time working with images. Scanning and digitizing is one thing we do on a fairly large scale.

To allow customers to see image detail at high resolution, we use Zoomify extensively. We generate the images in advance, using ZoomifyImage. OK, so we end up with huge numbers of files to store, but this is the 21st century and we can cope with it.

There are other options available that do similar things, of course. Just to mention a few: Deep Zoom, OpenZoom, and OpenLayers. One snag with some of these zooming capabilities is that they require browser plugins (Flash or Silverlight). Not only does this hurt users who don't have the plugin installed, but some platforms (OK, let's call out the iPad here) don't have any prospect of Flash or Silverlight.

However, modern systems do have HTML5 capable browsers, and HTML5 is really powerful.

A quick search finds CanvasZoom, which is a pretty good start. Given a set of Deep Zoom tiles it just works. I tried it on an iPad and it sort of works, not really doing touch properly. So I forked it on github (with ImageLoader for compatibility) and played with adding touch support.

It turns out the adding touch handling is pretty trivial. There's just the touchstart, touchend and touchmove events to handle. You want to call event.preventDefault() so as to stop the normal platform handling of moves in particular. The only tricky bit was working out that while you can get the coordinates for touchstart and touchmove from event.targetTouches[0], for touchend you have to look back at event.changedTouches[0]. So, poke the image and it zooms, poke and move and you can pan the image.

One thing I mean to do is to look at whether I can point it at a set of Zoomify tiles. I've already got lots of those, and just having one set of image tiles saves both processing time and storage. If not, I'll have to generate a whole load more images - I'm using deepjzoom which seems to do a pretty good job.

07 Nov 2011 8:51pm GMT

Tim Foster: Replacing the Application Packaging Developer’s guide

Over the last while, we've been writing a replacement for the venerable Application Packaging Developer's Guide.

I started at Sun in the European Localisation group, and writing tools to create (and sometimes hack) SVR4 packages for Solaris was one of the things I had to do quite a bit - I found myself consulting the Application Packaging Developer's Guide frequently. So, having the opportunity to help write its replacement was pretty cool.

Bart did a great job in defining the structure of the new book, and writing a lot of the initial content. Brock wrote some comprehensive content on package signing, SMF and zones, and I've been working on the guide since then, using some of the original documentation that Stephen wrote.

Unlike the previous guide, we have fewer examples in this guide. We feel that our man pages are better than the SVR4 packaging man pages were, and already contain useful examples. This guide is meant to compliment our man pages, not replace them.

The guide is a lot shorter than the old book - currently 56 pages, as opposed to the 190 pages in the document it replaces. Some of this is because of the fewer examples we have, but also we don't have to write about patch creation, complex postinstall or class-action scripting or relocatable packages. IPS is simpler than SVR4 in many ways, though there is a learning curve, which this book aims to help with.

There's more work to do on the book (as you can tell by the "XXX" markers that appear throughout) but what we've got so far is here: source code, and here, inelegantly rendered as a pdf.

If you use IPS, or have had experience in packaging software for Solaris previously, I'd be interested in hearing your thoughts on the content so far.

Updated: I've updated the PDF with the latest content as of today, and made sure there's a datestamp on the file, so you can tell which version you're reading. Changelog below (though I've not put the source back to the gate, yet, so these changesets will likely be collapsed)

changeset:   2576:6b14edfc38b5
tag:         tip
user:        Tim Foster
date:        Fri Oct 21 09:49:15 2011 +1300
description:
        Add another example to chpt14, republishing under a new publisher name

changeset:   2575:0cf619c409b1
user:        Tim Foster
date:        Thu Oct 20 15:51:22 2011 +1300
description:
        complete the SVR4 conversion example in appendix2

changeset:   2574:3a671202fd35
user:        Tim Foster
date:        Thu Oct 20 12:09:48 2011 +1300
description:
        Rewrite the var/share discussion in chapter 10
        Add macros and use them throughout the book
        Add a datestamp and logo to the cover page
        use hg identify for versioning
        add more text explaining why we don't "cross the streams"
        Add note on Immutable Zones to chapter1
        Nobody uses fax machines anymore, do they?
        Add text on pkg.human-version to chapter 3
        Add content for the user action
        Add content for the group action
        Introduce the constraints/freezing section of chapter 6
        Reference constraints/freezing section in chapter 7
        Describe XXX sections more clearly

Updated: I've added more stuff, replaced the links, but not yet pushed these changes to our external gate. Changelog below.

timf@linn[626] hg log -r 2577:
changeset:   2577:83de3ed97341
user:        Tim Foster
date:        Fri Oct 21 16:23:37 2011 +1300
description:
        Break up chpt5
        Move XXX for versioning disussion to chpt3
        Fix table of contents

changeset:   2578:df045bbafc98
user:        Tim Foster
date:        Fri Oct 21 17:19:22 2011 +1300
description:
        Write content for mediated links, with an example

changeset:   2579:1c3d87d950e6
tag:         tip
user:        Tim Foster
date:        Fri Oct 21 17:26:50 2011 +1300
description:
        Move XXX for versioning disussion to chpt3 (fix build)

Updated: Almost at the final version - there's a few small changes to make, but I've updated the links to the pdf of that version now, which hasn't yet been pushed to the gate. Too many commit messages to indicate what's changed unfortunately.

Updated: More minor changes and some style/branding tweaks. I've updated the pdf links above (2011-11-07:16:40:12-NZDT 8e2ee40e0bfb tip)


Filed under: IPS

07 Nov 2011 4:09am GMT

05 Nov 2011

feedPlanet OpenSolaris

James McPherson: The world has changed

If you have a look at various news services and Sun.COM right now, you'll see that the world has changed. I for one never really figured that Sun would choose the GPL v2 to license Java. I'm quite happy to be proved wrong. No, scratch that. I'm ecstatic to be proved wrong. It's going be damned interesting to see how the "Sun is evil because they won't GPL Java" crowd respond now. Especially IBM and HP. Sun has now placed under open source licenses its flagship cpu, its OS and now Java. I don't see an open-sourced AIX or HPUX. I don't see an open-sourced Power or HP-PA (let alone DEC Alpha!) chip. Come on IBM and HP, where are the actions to match the rhetoric? Technorati tags: topic:{Technorati}[Java], topic:{Technorati}[Sun Microsystems], topic:{Technorati}[GPL], topic:{Technorati}[Open Source], topic:{Technorati}[OpenJava]

05 Nov 2011 4:50am GMT

04 Nov 2011

feedPlanet OpenSolaris

Solaris Blog: Going to 11

And yes, this has also been much on our mind, too.

(Are you registered for next Wednesday's launch? Join us in New York, or watch on the web.)

04 Nov 2011 2:12pm GMT

Simon Phipps: ✈ T-DOSE in Eindhoven

I'm just setting out for Eindhoven in The Netherlands, where I will be delivering the keynote at 10:00 tomorrow (Saturday November 5th) at T-DOSE. I'll be considering how open source springs from a wider trend that also includes the Occupy* movement, and explaining why businesses should specify software freedom in their procurement activities. I'd love to see you there!


04 Nov 2011 12:45pm GMT

Simon Phipps: ☝ News Roundup

I just posted an open source news roundup over on ComputerWorldUK, covering UK government procurement, Microsoft and Samba, LibreOffice and IBM's MQTT. Take a look!


04 Nov 2011 12:44pm GMT

James McPherson: Submission to the Civil Unions bill Inquiry

This afternoon, alerted by a friend and as reported in this article, I sent the submission below to the Queensland Government's inquiry into Andrew Fraser's Civil Unions bill.

While I would greatly prefer that there be Marriage Equality for all, in this nation Marriage is part of the Federal Government's bailiwick and I'm not confident that either federal party will start listening to all their constituents on this matter.


To whom it may concern

    Submission to the Inquiry regarding the Civil Unions Bill


I am writing to express my support for Mr Fraser's Civil Unions
bill.

I have very dear gay and lesbian friends who have been in committed
relationships for many years. Under current law if one member of the
partnership is hospitalised, the other is prevented from being
recognised as the next of kin.

This is offensive and discriminatory because in many cases the natural
family members who current law elevates over the partner are unable
to accept that their relative is in a committed relationship, and who
then ignore the expressed wishes of the relative (as known by the partner)
in favour of their own opinion.

The situation is similar when it comes to superannuation, where gay
and lesbian partners must jump through significant extra hoops to be
recognised as a legitimate spouse or dependent. In particular, women
are widely acknowledged to be disadvantaged in later life through a
lack of sufficient superannuation funds. A recognised civil union will
obviate much jumping through hoops in order to prove the validity of
the relationship. This, in turn, will help prevent financial stress;
something which is acute even in non-LGBT couples when dealing with
the superannuation system.


Yours faithfully,
James C. McPherson 

04 Nov 2011 6:19am GMT

Solaris Blog: 11 Reasons Why Oracle Solaris 11 11/11 Isn't Being Released on 11/11/11

As I may have mentioned before, the Oracle Solaris 11 launch is this coming Wednesday, November 9th. Unsurprisingly, more than a few people have pointed out that just two days later, the 11th, is "11/11/11" -- a seemingly perfect day to launch a product named... well, you know.

So here's why we didn't go with that date. (Two of these are actually the real reasons.)

  1. Whoa -- we never thought of that!
  2. We just couldn't wait.
  3. Friday -- not a good day for a product launch.
  4. Armistice Day/Remembrance Day/Veterans Day -- not a good day for a product launch.
  5. Sony Pictures begged us not to release on the same day as the new Adam Sandler movie.
  6. We figured the launch parties would keep going for three days.
  7. Dyscalculia. Dysnumeria.
  8. Hoping to ride the coattails of the commemorations of Louis the Bavarian's defeat of his cousin Frederick I of Austria at the Battle of Gamelsdorf in 1313.
  9. "11" is actually 9--in octal. OK, that one's exceptionally weak.
  10. November 9 is Inventors' Day! (Which is held on this day because it's the birthday of actress Hedy Lamarr, who invented frequency-hopping spread spectrum signalling in 1942. I am not making this up.)
  11. "No... that's just what they'll be expecting us to do."

I hope that explains things adequately.

Anyhow, do join us in New York City at historic Gotham Hall if you can, or if you can't, be sure to tune in to the webcast.

On the 9th.

04 Nov 2011 2:42am GMT

03 Nov 2011

feedPlanet OpenSolaris

Damian Wojslaw: OpenIndiana and ntfs-3g

Update:
I have patch ready to change unhelpful on OpenIndiana, SmartOS and other illumos distributions mention of modprobe to suggest:
/usr/sbin/add_drv -m 'fuse 0666 root sys' fuse
Still have to find out, why the heck link was not created in the first place. :/


----------------------------------
I recently had to use ntfs formatted USB drive under OpenIndiana and run into a small problem.
I shamelessly used gparted to find which device maps to my usb disk. Then had following session with ntfs-3g binary:

ntfs-3g /dev/dsk/c2t0d0p1 /mnt/
The disk contains an unclean file system (0, 0).
The file system wasn't safely closed on Windows. Fixing.
fuse: device not found, try 'modprobe fuse' first

Both fuse and ntfs-3g are installed from repositories:
sfe origin online http://pkg.openindiana.org/sfe/
sfe-encumbered origin online http://pkg.openindiana.org/sfe-encumbered/


After some digging around and asking on IRC I decided that either no one had this problem or no one bothered to post solution. modinfo showed me that fuse module is loaded, after all:

258 fffffffff7edd000 e680 290 1 fuse (fuse driver)
258 fffffffff7edd000 e680 28 1 fuse (filesystem for fuse)


However ntfs-3g itself didn't say anything useful. It also suggested to use modproble, which is just ridiculous on OpenIndiana. I decided to truss it.


open64("/dev/fuse", O_RDWR) Err#2 ENOENT


One clue closed I downloaded spec file for fuse from http://pkgbuild.sourceforge.net/spec-files-extra/ and checked for %post step:

%post
( retval=0;
/usr/sbin/add_drv -m 'fuse 0666 root sys' fuse || retval=1;
[ "$retval" = 0 ] && ln -s /devices/pseudo/fuse@0:fuse /dev/fuse || retval=1;
exit $retval
)


I did check if /devices/pseudo/fuse exists (it did) and then created /dev/fuse link to it. Then ntfs-3g worked like a charm.

Cheers!

03 Nov 2011 2:13am GMT