12 Mar 2010

feedBSD Planet * BSD People

The Julipedia (Blog): New version of the monotone-server package in pkgsrc

Wow, it has been a long time... 5 years ago, I created the &lt;a href="http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/devel/monotone-server/?only_with_tag=MAIN"&gt;monotone-server&lt;/a&gt; package in pkgsrc, a package that provided an interactive script to set up a &lt;a href="http://monotone.ca/"&gt;monotone&lt;/a&gt; server from scratch with, what I though, minimal hassle.&lt;br /&gt;&lt;br /&gt;My package did the job just fine, but past year I was blown away by the simplicity of the same package in &lt;a href="http://www.fedoraproject.org/"&gt;Fedora&lt;/a&gt;: their &lt;tt&gt;init.d&lt;/tt&gt; script provides a set of extra commands to initialize the server before starting it up, and that is it. No need to mess with a separate interactive script; no need to create and memorize passphrases that you will never use; and, what's more, all integrated in the only single place that makes sense: in the init.d "service management" script.&lt;br /&gt;&lt;br /&gt;It has been a while since I became jealous of their approach, but I've finally got to it: I've spent the last few days rewriting the monotone-server package in pkgsrc and came up with a similar scheme. And this &lt;a href="http://mail-index.netbsd.org/pkgsrc-changes/2010/03/12/msg038844.html"&gt;new package&lt;/a&gt; just made its way &lt;a href="http://mail-index.netbsd.org/pkgsrc-changes/2010/03/12/msg038844.html"&gt;&lt;/a&gt;to pkgsrc-HEAD! The new package comes with what I think is a detailed manual page that explains how to configure the server from scratch. Take a look and, if you find any mistakes, inconsistencies or improvements to be done, let me know!&lt;br /&gt;&lt;br /&gt;In the meantime, I will log into my home server, rebuild the updated package and put it in production :-)&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-1599070089952057305?l=blog.julipedia.org' alt='' /&gt;&lt;/div&gt;</content>

12 Mar 2010 9:48am GMT

06 Mar 2010

feedBSD Planet * BSD People

The Julipedia (Blog): Introducing the ATF nofork branch

Despite my time for free software being virtually zero these days, I have managed to implement a prototype of what ATF would look like if it didn't implement forking and isolation in test programs. This feature has been often requested by users to simplify their life when debugging test cases.&lt;div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I shouldn't repeat everything I posted on the atf-devel mailing list regarding this announcement, so please &lt;a href="http://mail-index.netbsd.org/atf-devel/2010/03/05/msg000070.html"&gt;refer to that email for details&lt;/a&gt;. But I must say that the results look promising: the overall code of ATF is much simpler and also faster. (An execution I just tried cuts the run time of the ATF test suite from 1m 41s to 1m 16s.) Expect more simplifications and speed-ups!&lt;/div&gt;&lt;/div&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-1501363386005733392?l=blog.julipedia.org' alt='' /&gt;&lt;/div&gt;</content>

06 Mar 2010 8:48pm GMT

28 Feb 2010

feedBSD Planet * BSD People

Daemon Dancing in the Dark [FreeBSD Blog]: Do It Sudo

I've come across a couple sudo tricks during the past few days and thought I would pass them along. The first thing you need to know before using sudo is that you should use the visudo command to edit the sudo config file. This script does a number of useful...

28 Feb 2010 6:30pm GMT

22 Feb 2010

feedBSD Planet * BSD People

Hubertf's NetBSD Blog: Google Summer of Code 2010 NetBSD swcryptX Project Suggestion (Updated)

I've been thinking of a neat-o project for this year's Google Summer of Code:

Abstract: The goal of this project is to provide crypto acceleration by utilizing multiple CPU cores. The work is to extend the existing software-only "swcrypto" crypto driver and hook that up with NetBSD's OpenCrypto framework.

Overview of operation

The opencrypto(9) framework exists to coordinate hardware acceleration in NetBSD. Applications of the framework can be inside the kernel like the FAST_IPSEC IPsec implementation, or in userland like OpenSSL with the "cryptodev" engine. Crypto drivers can be realized in software or in hardware. Hardware drivers can be used to instruct e.g. the AMD Geode LX's AES block or a HIFN chip to perform cryptographic operations. Upon system startup, the crypto drivers at the opencrypto(9) framework, telling what operations they can perform. When an operation is required later, the framework will look which crypto device is currently not busy, and offload the operation to that device. Upon completion, the result is fed back to the application.

The following image illustrates the components and their interaction.

Limitations

Offloading the cryptographic requests involves some overhead. Data needs to be transferred to the hardware and back. On systems with a slow CPU, this overhead is relatively small compared to the operation speed of the CPU. On faster CPUs, the overhead becomes more of a burden, making the benefit of the crypto hardware negligible.

As examples, while a hifn(4) chip can provide worthwhile speedups on 500MHz and 1GHz CPUs, no performance win is experienced on a 2.4GHz CPU.

Proposal

The communication overhead involves data transfers over a PCI bus, which is of relatively low speed compared to today's modern CPUs. Preventing the data transfer is a worthwhile goal. In coordination with today's modern multi-core CPUs, using one or more CPUs solely for the purpose of crypto acceleration, a measurable improvement of crypto performance is expected. At the same time, no special hardware requirements beyond the CPU exist. This allows turning standard contemporary systems into fast crypto systems easily.

The following image illustrates the idea of interoperation between a CPU core that runs the kernel and application codes and three cores that are dedicated to crypto code.

Implementation Roadmap

This is where it gets fishy. ;) The existing opencrypto(4) framework probably needs to be make MP-aware at the same time, employing proper use of NetBSD's locking framework. (Already done) The existing swcrypto(4) needs to be adjusted for operation on multiple CPUs at the same time. A way to decide how many CPUs are dedicated to run swcrypto(4) instances. CPUs that run swcrypto(4) need to be taken out from the usual NetBSD CPU scheduling so that they are available exclusively for crypto.

Requirements

In no particular order:

Project Applications

Please follow the NetBSD Project Application/Proposal HowTo if you're serious to work on this project.

If you have any questions let me know, public discussion should be led on the tech-crypto@ list.

22 Feb 2010 10:57pm GMT

21 Feb 2010

feedBSD Planet * BSD People

Hubertf's NetBSD Blog: More news from NetBSD land: inside and outside the kernel

My inbox is still overflowing with NetBSD related news, so here is the next chunk for you:

I'm closing for today by pointing to three NetBSD-related events:

Have fun meeting the gang!

21 Feb 2010 2:20pm GMT

15 Feb 2010

feedBSD Planet * BSD People

Hubertf's NetBSD Blog: Setting up a PPPoE server with NetBSD

I had an urge to configure PPPoE today, but didn't have a DSL line handy to test it. So, with some help from martin@, I got to prod NetBSD into becoming a PPPoE server.

I've put my notes here.

Open issues:

I understand this method is only for one/few PPPoE connections. If more sessions are needed, this may not be the best option. It still served my purpose well. YMMV.

15 Feb 2010 9:56pm GMT

Daemon Dancing in the Dark [FreeBSD Blog]: Org-ing in Emacs

I have been using emacs since the beginning of computer time, way back in the mid-80s. My .emacs file has been carried around for almost as long. Each major rev causes me headaches, as I never know what archaic options are going to break things. And yet, I'm still learning...

15 Feb 2010 3:54pm GMT

12 Feb 2010

feedBSD Planet * BSD People

Hubertf's NetBSD Blog: Musing about git's object store efficiency

I'm currently looking at git to see what it can and cannot do, and one thing I've looked today is how effective the backing store mechanism is. To recall: CVS stores a list of patches between versions in a single file, and git stores each new revision in full in a separate file in the so-called object store. Is that an issue for NetBSD? Let's see;

One of the more frequently updated files is the i386 port's GENERIC kernel config file, which is at revision 1.963 right now. This means that since it's import into CVS, 963 different revisions have been made. In CVS, all those files are kept in a single GENERIC,v file. In git, this puts 963 files on the file system. A bit of a difference.

Looking at the space requirements for storing the repository data itself, the GENERIC,v file is 883,233 bytes[1]. Extracting all 963 versions from revision 1.1 to revision 963 results in disk space usage of 32,805,828 bytes[2,3]. And that's not counting the overhead of 962 inodes and the related directory bookkeeping.

In other words, the git model requires about 37 times the space that CVS does.

Sure the example file is not exactly one with an average number of revisions, and I know that git offers some more efficient storage methods via "pack" files, but investigating those is left as an exercise to the reader. :-)


[1] Obtained via rsync from cvs.netbsd.org:

% ls -la GENERIC,v 
-r--r--r--  1 feyrer  wheel  883233 Feb 12 16:57 GENERIC,v 

[2]

% mkdir extracted
% chdir extracted
% sh -c 'for i in `jot 964`; do echo $i ; co -p -r1.$i ../GENERIC >GENERIC-`printf %04d $i` ; done'

[3]

% cat extracted/* | wc -c
 32805828 

12 Feb 2010 5:26pm GMT

08 Feb 2010

feedBSD Planet * BSD People

Daemon Dancing in the Dark [FreeBSD Blog]: Tumblings, Jan 24

A couple of links posted to my Linux Tumblr last week: Wicked Cheap Hosting - some great web hosting deals, courtesy of All About Linux TermBuilder - a simple web app to build a commandline command...

08 Feb 2010 6:07pm GMT

07 Feb 2010

feedBSD Planet * BSD People

Daemon Dancing in the Dark [FreeBSD Blog]: Arched

So early in December, I went on a quest for a new Linux distro. It wasn't so much that I was unhappy with my openSUSE 11.0 installation, but I knew I was probably going to install the new 11.2 version and so I figured I would cast about to see...

07 Feb 2010 3:37pm GMT

05 Feb 2010

feedBSD Planet * BSD People

Hubertf's NetBSD Blog: Hackathon, February 19-22 2010

Aleksej Saushev writes on a bunch of lists:

We're running 13th Hackathon February 19-22 2010,
come and join us on IRC channel #netbsd-code at FreeNode (irc.freenode.net).
You may choose other ways to participate, if you find it more convenient.
See Hackathon page for updates: http://wiki.netbsd.se/Hackathon13


Goal

Fix as many bugs as possible, close as many PRs as possible.


Details

In previous years we have accumulated a lot of problem reports.
Some of them relate to no longer supported branches (e.g. 2.0)
and were occasionally fixed during regular code work.
Some of them relate to hardware developers don't have access to.
Some of them may be too easy to fix, but noone looks at that part
of code (e.g. documentation).
Some of them relate to packages already removed or heavily reworked.
Some of them relate to packages in exotic environment (Solaris, Interix)
and developers cannot test if the bug is present or not.

You can find more in PR database at http://netbsd.org/support/query-pr.html


More specifically, David Holland maintains categorized lists of PRs at

  http://www.netbsd.org/~dholland/buglists/

of which "Wanted for 5.1" 

(http://www.netbsd.org/~dholland/buglists/51-WANTED.html)
and "Stuck" (http://www.netbsd.org/~dholland/buglists/STUCK.html)
are of particular interest.

Another hot area is pkgsrc 
(http://www.netbsd.org/~dholland/buglists/pkgsrc.html).


IMPORTANT NOTE: You don't have to be programmer to be able to help us,
you can help us with generic testing support.  

Come, bring your favourite booze, and join in!

05 Feb 2010 11:00pm GMT

02 Feb 2010

feedBSD Planet * BSD People

Daemon Dancing in the Dark [FreeBSD Blog]: Linux Love Links

A few things I have recently thrown down on my Linux Love Tumblr blog, which I use for quick little Linux links and notes: 7 Best Linux Apps for Ripping CDs and DVDs | Maximum PC 50+ Ultimate Collections of Planet Wallpapers TermBuilder: a graphical Linux command line generator...

02 Feb 2010 8:46pm GMT

Hubertf's NetBSD Blog: Turn your ALIX into KITT (Updated)

Drop the following script into /etc/rc.d/alixkitt, put "alixkitt=yes" into /etc/rc.conf and enjoy:

#!/bin/sh
# Turn ALIX2c front LEDs into running LED
# http://www.gifninja.com/Workspace/59f3f22e-5f40-4937-936c-1dc0d6fbe690/output.gif
#
# Copyright (c) 2010 Hubert Feyrer <hubertf@NetBSD.org>

# PROVIDE: alixkitt

$_rc_subr_loaded . /etc/rc.subr

name="alixkitt"
start_cmd="alixkitt_start"
stop_cmd="alixkitt_stop"
pidfile="/var/run/${name}.pid"

alixkitt_sleeptime=1.0          # 0.5

alixkitt_setled()
{
        gpioctl -q -d /dev/gpio 6 $1
        gpioctl -q -d /dev/gpio 25 $2
        gpioctl -q -d /dev/gpio 27 $3

        sleep $alixkitt_sleeptime
}

alixkitt_start()
{
        touch $pidfile
        (
                alixkitt_setled 0 0 0

                while [ -f $pidfile ] ; do
                        alixkitt_setled 0 1 1
                        alixkitt_setled 1 0 1
                        alixkitt_setled 1 1 0
                        alixkitt_setled 1 0 1
                done
        ) &;
        echo $! >$pidfile
        echo Started pid $!
}

alixkitt_stop()
{
        if [ -f /var/run/alixkitt.pid ]; then
                read pid <$pidfile
                echo Stopped pid $pid
                kill $pid
                rm $pidfile
                alixkitt_setled 1 1 1
        fi
        exit 0
}

load_rc_config $name
run_rc_command "$1"

P.S.: There's still a bug left in that stopping immediately re-starts the process. Got a patch? Update: Axel Scheepers suggested doing the while-loop only as long as the PID-file exists. Changed above.

02 Feb 2010 10:18am GMT

01 Feb 2010

feedBSD Planet * BSD People

Daemon Dancing in the Dark [FreeBSD Blog]: On Being Persistent

Those of us who have multiple hard drives in our computers will inevitably boot up one morning to find the naming scheme for these drives has changed. What was once /dev/sda is now /dev/sdb and vice versa. Your computer won't boot and fsck complains about an uknown or mismatched filesystem...

01 Feb 2010 2:28pm GMT

31 Jan 2010

feedBSD Planet * BSD People

Hubertf's NetBSD Blog: Toggling Alix2c LEDs

I've found that the three LEDs at the front of a PC Engines Alix2C board can be toggled on and off via their attachment to the GPIO bus. Here are the commands for this:

left: gpioctl -d /dev/gpio 6 2
middle: gpioctl -d /dev/gpio 25 2
right: gpioctl -d /dev/gpio 27 2

31 Jan 2010 11:50pm GMT

Hubertf's NetBSD Blog: Unfilling my inbox: NetBSD news from the past few weeks - ACPI, NUMA, Xen, and more

Herre are some more things that I've caught in my inbox for too long, and I'm finally finding some time to sum them up here:

So much for today. I still have a bunch of news items in my inbox for next time, but let's call it good for today.

Unrelated, I've been playing with git a bit over the past few days, and wile I have a number of questions building up (which will be subject to tech-repository or so), what I can say today is that the speed of "git pull" with NetBSD's git repository and my 1MBit DSL line reminds me a lot of the times when I used SUP with my 56k modem - it took forever, too. :-(

31 Jan 2010 4:54pm GMT