fidzu : Perl

It's eighteen months since I wrote "Why Corporates Hate Perl" and it's worth pointing out that the company I discussed in that article which was dropping Perl in favour of PHP and Java is still employing as many good Perl programmers as it can find.

I talked in that article about some rather unsubtle social engineering that had been going on at that company. Management had started to talk about Perl as a "legacy language" in an attempt to persuade the business that they really don't want their applications written in Perl. That doesn't seem to have been as successful as I feared it would be.

But there's another kind of social engineering that I've seen going on. And this is happening at the developer level. I've lost count of the number of times I've been sitting in meetings with developers, discussing ways to improve the quality of crufty old Perl code when someone throws in the (more than half-serious) suggestion that we should just rewrite the whole thing using Rails or Django.

There seems to be this idea that switching to a new framework in a new language will act as some time of magic bullet and that suddenly all of our problems will be solved. There are so many ways that this perception is flawed, Here are my two favourites.

1. The current system is old and crufty not because it's written in Perl, but because it was written by people who didn't understand the business requirements fully, didn't know their tools particularly well or were under pressure to deliver on a timescale that didn't give them time to design the system properly. Often it's a combination of all three. Given the time to rewrite the system from scratch, of course it will be better. But it will be better because the business is better understood and tools and techniques have been improved - not because it's no longer written in Perl.
2. Frameworks in other languages are not easier to use or more powerful than frameworks in Perl. Anything that you can do with Rails or Django you can do just as easily with Catalyst. It's using a framework that's the big win here, not the particular framework that you use. Sure, if you're a Ruby house then using a Ruby framework will probably match your existing developers' skills more closely but if your current system is written in Perl then, hopefully, you have a team of people with Perl skills and that's going to be the language you'll want to look at.

I'm tired of Perl being seen as a second-class citizen in the dynamic languages world. I freely admit that there's a lot of bad Perl code out there (I'll even admit to writing some of it) but it's not bad Perl code because it's Perl, it's bad Perl code because it's bad code.

This is what the Perl marketing initiative is for. We need people to know that Perl is not the same language that they stopped using ten years ago. Modern Perl can compete with any of the features of any other dynamic language.

By al means, try to get the time to rewrite your crufty old systems. But rewrite them in Modern Perl. You'll enjoy it and you'll be really productive.

p.s. I should point out that I'm not talking about any specific client here. This is based on conversations that I've had at various companies over the last couple of years and also discussions with many developers in many pubs.

12 Mar 2010 9:52am GMT

Yes, threads suck. And in several different ways. Slow, memory-bloating, etc etc etc.

However, threads work.

Installing them from the CPAN on Windows, Mac and Linux works.

Using them as they are intended to be used works on Windows, Mac and Linux.

Integration with Wx works on Windows, Mac and Linux.

And using them to saturate a 4 CPU core development machine works, without resorting to having both them AND external processes.

And they work well enough that nobody has yet had an itch strong enough to step up and replace our task management code with something else that can support more than one CPU (and since our task system is derived from Process.pm, it is specifically designed to make it easy to use alternative backends).

So threads work, but they suck. Or at least, interpreter-copy sucks.

And in an IDE scenario where your process needs to load 50-100 meg of code just to drive the cooler IDE functions they suck even harder.

Fortunately, Padre has a rather forgiving attitude to things sucking.

We would rather have someone commit something that works and sucks, than not have it committed at all. And Padre is full of all kinds of features that work but suck. And slowly, gradually, they suck less.

The most recent couple of releases have come with stability warnings due to the arrival of our second-generation "Slave Driver" threading model.

The Slave Driver mechanism attempts to specifically contain and reduce the problems associated with threads.

During startup, we load the minimum number of modules required to conduct communication across threads, and then immediately spawn off a master thread which will remain unused while our main thread continues onward and loads up as much code as it likes.

Later on, when we need to do background work, this master thread is then further cloned into a slave thread which will bloat out incrementally, loading only the code it needs to execute the task.

While the slave driver mechanism itself landed a few weeks ago, the final step of pushing the master spawn point up into the start-up code just landed today.

The result of this is a change in our per-thread cost from 34meg per thread to about 20meg per thread (for a total reduction in memory in a low-usage case from 90meg to 60meg).

If not for the fact that loading Wx is an all or nothing proposition, we could probably cut this in half again. And while a collection of half a dozen 35meg threads (about the maximum we are likely to need to saturate 4 CPU cores) in a thread pool is a nasty amount of RAM, even for an IDE, half a dozen 10meg threads is a lot closer to a tolerable memory cost.

And if we can drop this further to around 5 meg, we get close to the memory cost of a forking/process model, which is the only other parallelism model available in the short term that supports many cores transparently.

12 Mar 2010 5:26am GMT

In short, there was just too much copy and paste. It wasn't really trivial, either. Here's an example of code that appeared in both Pod::Weaver::Role::Plugin and Dist::Zilla::Role::Plugin:

for my $method (qw(log log_debug log_fatal)) {
  Sub::Install::install_sub({
    code => sub {
      my ($self, @rest) = @_;
      my $arg = _HASHLIKE($rest[0]) ? (shift @rest) : {};
      local $arg->{prefix} = '[' . $self->plugin_name . '] '
                           . (defined $arg->{prefix} ? $arg->{prefix} : '');

      $self->weaver->logger->$method($arg, @rest);
    },
    as   => $method,
  });
}

This is gross for a few reasons, but I think that its basic grossness should be obvious. I didn't want anyone to have to copy this around. The replacement is really nice:

has logger => (
  is   => 'ro',
  lazy => 1,
  handles => [ qw(log log_debug log_fatal) ],
  default => sub {
    $_[0]->weaver->logger->proxy({
      proxy_prefix => '[' . $_[0]->plugin_name . '] ',
    });
  },
);

We say, "I'm a plugin of something with a logger. If I need to log anything, I'll ask for a proxy to that logger, and my proxy will have a custom prefix."

This proxy is a Log::Dispatchouli::Proxy, which closely emulates much of Log::Dispatchouli, but with some bits localized and some bits off limits. For example, that proxy_prefix can't be changed after the fact, but we could turn debugging on or off without affecting the parent logger.

Because I'm using proper delegation again, I can make things lazy again, and I can make it easy to tweak the behavior of one plugin for testing purposes.

This logging work has taken more time than I expected, but I think it will really pay dividends. I hope to get the new logging code documented and all the relevant updates deployed to the CPAN tomorrow. For now, I'm beat and I'm going to bed.

12 Mar 2010 4:15am GMT

Sewi, Phaylon and the Foswiki developers have already written very good posts about CeBIT and the Perl booth in English and Renee in German (see all posts listed on the Perl::Staff page and on our wiki. (though I don't know what interesting discussions Sewi mentions as neither of us could actually talk in the evenings...)

I must say a big, big public thanks to Sebastian Willing (Sewi) and Tanja who took me in their house occupying their living room for a whole week during CeBIT. Not only made that the whole trip affordable but it was really nice to be with them and to get to know each other a little bit more.

I also would like to thank the people of Perl::Staff for being at the Perl booth. First of all to Renee Baecker who prepared most of the new material for the fair and who took me from Frankfurt and back again. Thanks to Torsten Raudssus, (aka. Getty or the man with the wings), who did not have a blog a week ago and only reads the recent upload feed of CPAN. If you'd like to get his attention, you'll need to write the text in the documentation of a module and upload to CPAN.

Thanks also to Thomas Fahle, Robert 'phaylon' Sedlacek, Bernhard Schmalhofer (barney), Oliver Kr\x{00fc}ger and Michael Daum, the last two from the Foswiki team.

Our good neighbour Andreas \x{201c}ads\x{201d} Scherbaum, the friendly Postgres guy helped us a lot both before and during the event and of course Britta Wuelfing from Linux Magazin who was our contact person before and during CeBIT.

The materials we used and distributed were sponsored by TPF - The Perl Foundation, by YEF - YAPC Europe Foundation and we also got a bunch of beermats from EPO - Enlightened Perl Organization.

The postcards with the few projects and the list of Perl Events that we used in FOSDEM were still very good. The community business cards were also working in some cases but less good than in FOSDEM and IMHO the tuits were almost unused. Both because of some language barrier and because that did not fit the type of people visiting CeBIT and our booth.

The big blue promotional material Renee prepared was used well but I think it would have been more useful if it was in German. For future events we should have similar things in several languages. The content of those pages can be found on the wiki page"

The beermats were quite useful - in many cases after some discussion, I asked people if they drink beer. Most of them did not understand what a stupid question. Of course they drink beer. They then liked very much the beermats. We also had a special event in the evening of the 3rd day when we there was a party just next to us. People got free beer and we supplied the "open source beermats, based on Perl". People liked it very much.

We managed to draw a smile on the faces of many visitors which I think is one of the big things we can do. Create positives feelings associated with Perl.

I think that made us a few more friends. Well, maybe except that Python enthusiast who used the opportunity of the beermats to give the standard we-use-python-and-hate-perl speech. That reminds me that we actually had a few Python programmers at the stand. They initially did not want to hear anything saying "We are python programmers, we don't care about Perl" but then we could lure them to our computers and show them some modern perl code that impressed them. So there are Python programmers that are open minded and the appropriate amount of friendliness can bring them closer. This probably is related to what Robert Sedlacek wrote in the Other dynamic languages section.

The catch man

When you walk around in area frequently visited by tourists you often see people standing in-front of the restaurants and various other establishements and trying to direct people to enter the place. In a large part that was what I was doing. As people passed by our stand I gave them either a postcard and when we ran out of it some other promotional material. As they looked at the material I soon asked if they speak English. That surprised them a bit and many said they speak German which just gave me the opportunity to point at one of the other Perl::Staff members standing near-by and telling them they speak German.

That was both good as it freed me to go to the next person but it was also better for the visitors that they could speak in their mother tongue. I think that's important. So while we are preparing to attend more events, and I'd be happy to visit more such events and kick-start the local Perl::Staff group but it should be just that. Kick-starting the local group as the visitors will prefer to talk in the local language anyway.

We have listed many many tech events on our wiki page but I am sure we left out a lot more. We are also looking for people who will want to participate in promoting Perl. As other have already mentioned it is both hard work but it is also fun. It is very different from sitting in a cubical and coding or even talking to fellow Perl hackers on a YAPC but it can be rewarding both personall - especially in the evening when you can finally rest ... ;-) - but also professionally.

Pictures

BTW There are some pictures as well about Perl on CeBIT.

12 Mar 2010 1:53am GMT

During and the "Open Source Days 2010" in Copenhagen there was a Perl 6 track, and also a hackathon that stretched out three more days after the conference.

I arrived on Saturday afternoon (which happend to be the last day of the conference) and thus missed all of the talks and the more public part of the hackathon.

Module Loading

On Sunday we discussed module loading extensively. Martin Berends had prepared that topic, and drove the effort by asking the right questions.

Discussions

Modules are complicated beasts in Perl 6. They not only have a name and a version as in Perl 5, but they can also have so-called authorities, which handle the case when different programmers write modules with the same name. Also unlike Perl 5 a module can be installed with different versions, and the programmer can either request a particular version or the highest version available.

Also they have Unicode and case sensitive names, which Perl 6 must support even if the underlying file system does not provide both features.

Although it doesn't really sound like it, this combination of requirements make it incredibly hard to implement a module loader that is both correct and efficient. Previously all discussions on the mailing lists and IRC channels ended in huge threads full of bikeshedding and even more feature ideas.

We were well aware of these requirements before the hackathon, and also aware of the fact that we have no chance of implementing all that in a reasonable time frame. So the first step was to decide what subset of features to implement, and how that could be achieved.

We decided not to implement Unicode emulation/mapping, and partially ignore authorities for now. Still we leave some room for wriggeling in the mapping from module name to file name: a module My::Module can live in the file My/Module.pm or in My/Module.ignored.pm. That way several modules with the same name but different versions or authorities can be stored in the same module repository.

This also implies that the Perl 6 compiler has to actually read and parse those files to find out which file to load. In a later stage the compiler will write a cache file to store the mapping from file name to module name(s), verisons, authorities, a timestamp and probably also dependencies plus timestamps (in order to know when to recompile a module).

Once this caching mechanism is implemented, it solves the Unicode problem mostly for free, because it will store the full Unicode module name and ASCII file name in the cache, and makes it available to the module locater.

After reaching an initial consensus, the discussion went on to other topics: can we use the existing CPAN infrastructure to actually distribute our modules? The conclusion was that we most likely can use (with very little patching) the PAUSE and mirroring infrastructure, but we likely have to write our own indexing and searching facilities and also a completely separate module installer.

Implementation

I wrote a very simplistic prototype of a module locater in Perl 5, which Jonathan mostly translated to NQP on Monday, and actually plugged it into Rakudo's module loading facility on Tuesday.

So now you can actually have multi modules with different versions, and load the one with the highest version - requesting a particular version is not yet implemented, but hopefully now a mere SMOP.

Other matters

Martin Berends wrote up the results of our module discussions, and spent the rest of his hacking time on a foreign function interface.

Not everybody worked full-time on module things; baest worked on some builtin functions and (most of the time) number handling. Carl Mäsak spent some time on tardis, his time travelling debugger. He also spent quite some effort on getting enums back into Rakudo, with occasional assistance from Jonathan.

Many of us fixed some bugs that popped up, and also found new bugs

RT Queue

The perl6 RT queue has grown to such a size that it was both scary and hard to use at all. On Sunday it peaked around 725 open tickets.

I did rough sweep over most of the tickets, identifying those that were either fixed (but not yet closed), superseded by changes to the specification or actually not bugs at all (spam and false reports), or could be closed by simple changes to Rakudo.

That way I closed about 80 tickets, and identified another 40 or so tickets that are actually fixed, but need test coverage before being closed.

Personal notes

This was the first Perlish event to which I have traveled, and it was just awesome. I met friends in "meat space" that I previously only (or mostly) knew over the Internet, and found them to be just as I had experienced them from the distance: friendly, witty, full of good jokes and bad puns, curious, relaxed and all in all a very enjoyable company.

We enjoyed the hospitality of Jonas Brømsø Nielsen and the OSD folks who did everything to make our stay pleasant: prepare the arrival with maps and tips about the public transport system, booking and funding (!) our accommodation, inviting us to lunch or dinner now and then, and being available whenever questions or problems arose. Thank you!. The March release of Rakudo will be named Copenhagen for very good reasons.

10 Mar 2010 7:59pm GMT

The latest heavy project at $work makes me miss C a bit. There's a lot of Moose objects, KiokuDB, MooseX::Getopt (which is evidently much lighter than I thought) and more. To display all the entries I have in a KiokuDB SQLite database, it takes roughly a second and a half!

Profiling with NYTProf 3.0 (FTW!) shows that Moose takes the most. I don't want to use Mouse for various reasons which I won't go into. Moose::Compile, MooseX::Antlers, nothing near production yet (but I'm keepings my fingers crossed). I tried to rewrite the MooseX::Getopt part and use Getopt::Long manually, but it only speed it up by 30ms. Oh, and yes, everything is immutable. Requiring KiokuDB::Navigator only in the part that loads the navigator gained me 300ms.

I can start rewriting everything with clean Perl objects. It could take a while, but I could get it done. However, that would be losing all the fun and power I get from Moose. Instead I thought of a different idea.

Since I want to integrate this with another database backend (which might be on a remote server) anyway, the speed will slow down a lot because even a small request for display the local database entries would make a trip to the remote database, so even without Moose, the penalty would be great.

I thought of putting everything in a daemon, which would compile and load all the code. The client would be simple code which just asks the daemon to run things. The runtime penalty can't be too big, so I think it would work out. The daemon could fetch the remote data once in a while or per command. That's about it.

I might try to rewrite it without Moose and see the differences. That would be interesting to note in the lectures I'll be giving on Moose next week.

10 Mar 2010 11:43am GMT

Problem:

I've been upgrading some systems at work recently. Using local::lib so I can test it first without worrying about rolling back being a pain (which it could be with our current build system).

We've had most of our CPAN modules installed for several years and haven't had the need/time to upgrade them. But with the advent of Catalyst 5.8 using Moose and other great projects we'd like to get involved in, such as Gitalist I bit the bullet and started installing into my local::lib so I could then run our tests against it, but then I hit a few problems....

Now remember when I said we had old module? - I'm talking some that haven't been upgraded in 4+ years (they only get upgraded when a newer version is set at a prerequisite).

What I found was although most modules passed their tests, there were a few that wouldn't, and this was because the package either didn't specify the dependency, didn't specify the dependency version, or required a module that now had a broken version on CPAN (and actually didn't need the module anymore because they'd coded around it!).

Solution:

I was glad to find all of the packages which had these issues were on github. So I:

1. 
   
2. click fork
   
3. clone to my local machine
   
4. Edited Makefile.PL, Changes and set a new $VERSION
   
5. push, back to git up
   
6. click 'send pull request' (to inform the author)
   

Out of 4 distributions patched, I've already had two confirmed as merged in, one release to CPAN and this is all within 24 hours!

I've even submitted a minor one line doc patch to another module this way and it was on CPAN that afternoon.

Git (and github) really does make supplying (and accepting) patches very easy, so the next time you find an issue, see if the distribution has a repository and create a patch, it really doesn't take much time at all.

I've almost finished migrating all my own CPAN modules to Github, so if you'd like to patch those please fork from http://github.com/ranguard/.

Thanks:

Of course all of this wouldn't be possible without the great work people do in maintaining their modules (accepting and uploading patches for example!) in the first place - so once again thanks to the Perl community as a whole, you make my work FUN!

10 Mar 2010 9:14am GMT

I've been following stories about Roger Ebert's new voice, which a company has made so that they can apparently plug it into his Mac. In his appearances on camera, the voice he's been using is the Mac "Alex" voice. What other voices does your Mac have? Here's a Perl one-liner to play them.

From the Terminal window, paste this in at the prompt:

perl -le'for (@ARGV){m{/(\w+?)\.SpeechVoice};' \
  -e'$_=$1;s/([a-z])([A-Z])/$1 $2/g;' \
  -e'print qq{say -v "$_" This is $_};}' \
  /System/Library/Speech/Voices/* | sh -x

On my Mac, "Organ" isn't found, but I don't know why.

10 Mar 2010 5:45am GMT

The last 24 hours or so have been really busy. Today was Martha's third birthday (wow, can you believe it?) and I've had a number of incredibly annoying bugs at work. I won't get into the details right now but one of them was an MSIE bug that I solved by \uXXXX-only double-JSON-encoding a structure. The other was fixed by disabling Etag headers. Blaaaagh!

Despite this, I found some time for working on Dist::Zilla improvements. I did a pretty significant amount of work, actually, but if I paste the results, it will not look very impressive:

[DZ] beginning to build Dist-Zilla
[DZ] guessing dist's main_module is lib/Dist/Zilla.pm
[DZ] extracting distribution abstract from lib/Dist/Zilla.pm
[@RJBS/Classic/ExtraTests] rewriting release test xt/release/pod-coverage.t
[@RJBS/Classic/ExtraTests] rewriting release test xt/release/pod-syntax.t
[@RJBS/PodWeaver] [@Default/Name] couldn't find abstract in lib/Dist/Zilla/Tester.pm
[@RJBS/PodWeaver] [@Default/Name] couldn't find abstract in lib/Dist/Zilla/App/Tester.pm
[DZ] writing Dist-Zilla in Dist-Zilla-1.100680
[DZ] writing archive to Dist-Zilla-1.100680.tar.gz

The change, here, is that Pod::Weaver's plugins can now log as a "second order" set of plugin data. This sort of worked, in theory, with previous implementations of logging, but it's much simpler now. I've ripped out a lot of the logging code from Dist::Zilla, eliminating the illusion that you could use something other than Log::Dispatchouli to replace its logging. At the same time, I refactored some of Log::Dispatchouli's methods to make it more flexible, but not (I hope!) more complex. I don't think much will come of it, beyond what I've just shown above, but it's still useful. It will also let me easily do something like this:

$self->log_debug(
  { prefix => $self->prefix_wrap },
  $multi_line_log,
);

...to produce...

[PluginName] <<< This is a multi-line
[PluginName] ||| log entry that will be
[PluginName] ||| marked with some sort
[PluginName] >>> of log-item grouper.

The real win here is "less code." I had to copy and paste a five or six line hunk of code around a few times, and I think that will get refactored later. Still, things are very simple, now, and it will make testing the logging of plugins very easy.

Next up: actual testing.

I hit one stumbling block with my previous attempt to make it possible to build a dist found in another directory. Because of the way things worked, I ended up getting dist tarballs with contents like ~/User/rjbs/code/cs/Whatever/lib/...instead ofWhatever-1.234/lib/...`. I fixed this by reverting the directory absolutizing. I'll either have to do a more comprehensive survey and fix of the code or I'll just more aggressively add localized chdirs. That's fine, too.

So, next up: I start to really test things. First up, I'll make the existing dz1.t and dz2.t tests actually test things. After that, it will be time to start testing each plugin. That should be... well, awful. It will be nice to have the tests, though!

10 Mar 2010 4:52am GMT

These links are collected from the Perlbuzz Twitter feed. If you have suggestions for news bits, please mail me at andy@perlbuzz.com.

• What's going on with San Francisco Perl Mongers? (blogs.perl.org)
• Contributing is easy: A first-timer talks about adding to CPAN (blogs.perl.org)
• Help keep the world safe from SQL injection: (perlbuzz.com)
• How to learn how to get help in Perl (use.perl.org)
• The Perl Foundation wants your input on grant proposals for 2010 (news.perlfoundation.org)
• Why Ruby is prettier and Padre changes the Perl community (use.perl.org)
• Manage your Perl modules with git (effectiveperlprogramming.com)
• Call for papers for YAPC::NA 2010 in Columbus. Theme: Modern Perl 5 (news.perlfoundation.org)
• Famous Perl One-Liners Explained part 5: Text conversion and substitution (catonmat.net)
• The Italians like ack, too (cattlegrid.info)
• Bash completion for perldoc (blogs.perl.org)
• cpanminus, the new CPAN shell superstar (marcus.nordaaker.com)
• RT @chromatic_x Kudos to *many* active #perl 5 porters for the successful 5.11 release series: (ur1.ca)
• Testing a DB-intensive app (blog.urth.org)
• More news on the TPF wiki overhaul (use.perl.org)
• Do you still program in Perl? (reddit.com)
• Benchmarking is for losers, Profiling rulez! (blog.urth.org)
• Damian Conway's been posting a series of columns on vim: (blogs.perl.org)
• Testing and looging with Dist::Zilla (rjbs.manxome.org)
• Safe.pm 2.25 fixes security hole (blogs.perl.org)
• Serving production web requests with the Catalyst development server (blog.afoolishmanifesto.com)

09 Mar 2010 4:47pm GMT

Howdy,

I am working on the application for The Perl Foundation and Parrot
Foundation to participate in Google Summer of Code 2010 [0]. GSoC is a
program where Google funds eligible students to hack on open source
projects for a summer. It is a great opportunity for the students and
the communities that mentor them. You also may be interested in this
summary of our involvement last year [1]. Our application will be
submitted by the end of this week.

Please join us in getting prepared for this year. There is a page for
possible mentors to volunteer [2]* as well as a page for project ideas
[3]. If you would like to help with the wiki, our main GSoC page [4]
is the best place to start. You are also invited to join our mailing
list [5] and come ask question in #soc-help on irc.perl.org .

Thanks!

Duke

[0] http://socghop.appspot.com/
[1] http://google-opensource.blogspot.com/2009/10/perls-of-wisdom-perl-foundation-parrots.html
[2] http://www.perlfoundation.org/perl5/index.cgi?gsoc_mentors
[3] http://www.perlfoundation.org/perl5/index.cgi?gsoc_2010_projects
[4] http://www.perlfoundation.org/perl5/index.cgi?gsoc
[5] http://groups.google.com/group/tpf-gsoc

* If you listed yourself as a mentor last year and you are not
interested this year, please remove yourself from the page.

--
Jonathan "Duke" Leto
jonathan@leto.net
http://leto.net

09 Mar 2010 1:12am GMT

Ever wonder why bash closes if blocks with fi? This practice was inctroduced in Algol 68, a language that Perl 6 was accused of reinventing yesterday on the perl6-language list.

Curious, I went to the Wikipedia article to read up on Algol 68.

"Successor." "Wider scope of application". "More rigorously defined syntax and semantics". Sound familiar?

Oh dear. ☺ We even have the 'do little to make the compiler writer's task easy' meme in Perl 6...

Besides that, there's all these other little parallels, such as

• Algol 68 seemingly playing with words (they borrowed the term 'gomma' from Finnegan's Wake, but the feature it denoted got scrapped in a 1973 revision),
• something junction-like called 'multiple value',
• a whole heap of values for different forms of nothing and undefinedness,
• a newly-invented grammar formalism, and
• a general feeling of deep ambitiousness and a desire to get things right.

So, there are deep similarities between Algol 68 and Perl 6. There's not much to say to that, except perhaps "huh".

If there's anything in it all that's uplifting though, it's the second paragraph of the article:

If that's not spot on for Perl 6, I think it will be in a decade or so.

08 Mar 2010 10:01pm GMT

I feel like I'm making enough progress, right now, to make a grant update every day. I don't know how long this will last, but I'm glad that it's going so well, so far.

I got a lot of preliminary improvements done to logging, mostly described in yeserday's post. They scrunched the default output way down to fit on the screen and be readable. Now that I had logging working, it seemed like time to bite the bullet and really get to work on testing.

I made some improvements to App::Cmd::Tester, trying to capture the exit code of applications that call exit. I'm not entirely happy with how it works -- it interferes with Test::Builder sometimes, it seems. Still, it's better than nothing. I made a few other changes, too, making the App::Cmd::Builder easier to extend and providing more information back in the results. For example, you can now say:

my $results    = test_app(AppClass => \@my_argv);
my $app_object = $results->app;

This is really useful when the app object is a Dist::Zilla::App, because you can do things like:

my $app_object = $results->app;
my $zilla      = $app_object->zilla;
my $build_dir  = $zilla->built_in;

Obviously, this makes testing Dist::Zilla a lot easier.

I've also written a subclass of Dist::Zilla just for testing, which will copy the source material to a temporary directory and then build to another part of that temporary directory. Then the programmer can inspect the files written to disk, the state of the Dist::Zilla object, and more. I also hope to write a simple way to ask, "Did anything alter the source tree?" (That will often constitute a bug, but not always.)

Unfortunately, I've exposed one bug and one (recent) design flaw. The bug is something I've basically known to be there all along, and hasn't mattered at all for normal use. In short, you can't build a dist unless the dist root (the place with the dist.ini file) is your current working directory. If you try, things break, because too many things work with relative paths and never properly make them absolute. This, I hope, will not be very hard to fix. Fortunately, I can work around it with judicious use of File::chdir for now.

The more annoying problem is that the logger design is a problem. I still like Log::Dispatchouli and plan to keep using it as the default logger, but the way the existing logging framework works is just too leaky. It acts like the logger can be replaced, but it clearly can't, and even doing things like logging everything to a test object isn't easy enough. I have a few possible solutions to this, ranging from the brutal-but-easy to the complicated-but-maybe-more-useful.

I think I'll start with the latter, assuming that I won't actually need the added complexity. At any rate, I think it's extremely likely that before the end of the week I will have a nice simple way to test Dist::Zilla's behavior in automated tests without copying and pasting a lot of ugly code. This is really exciting, because it means I can start writing tests for Dist::Zilla. It's also really scary, because I'm pretty sure that writing tests for Dist::Zilla is going to be a big drag. That's okay, though, because it's what I'm getting paid to do.

Finally, I thought I'd note one other really exciting thing. Dist::Zilla has a lot of dependencies, mostly because I've tried to factor out any code that can be useful elsewhere, but also because I've tried to use existing tools when they exist. Now that I'm trying to get these improvements made to Dist::Zilla, I'm finding that a lot of them need to go in external libraries. That means that this grant to improve Dist::Zilla is also going to improve App::Cmd, Log::Dispatchouli, Pod::Weaver, and probably a bunch of other libraries. That makes me pretty happy about the decisions I've made so far.

08 Mar 2010 3:02pm GMT

I am happy to announce that the Haifa Perl mongers are going to have a meeting on the 15th March in the offices of Qualcomm in Matam, Haifa organized by Shmuel Fomberg. On the agenda is Erez Schatz How to Talk to Newbies and Yaron Meiry (aka Sawyer) Moose - A postmodern metaclass-based object system for Perl 5 The meeting will start at 18:30.

For more details please see the announcement and/or contact Shmuel.

The regular Rehovot Perl Monger meeting is going to take place on 16th March in the Weizmann Institute. Yaron Meiry (aka Sawyer) is going to give a talk about Moose - A postmodern metaclass-based object system for Perl 5

For more details please see the web site of the Rehovot Perl Mongers.

08 Mar 2010 1:39pm GMT

Lately I've been learning ExtJS (which I might write about separately sometime) to try and create a UI for a small application I'm writing for a friend.

The application uses Dancer and KiokuDB. They're both easy to work with and I spent little time working on the backend. The problem lies in the frontend, where I need some UI. I've chosen ExtJS because it has built-in widgets.

I've played with tutorials, general documentation and (tried to) read the not-as-friendly-as-I-thought API. I was able to create a grid, a form and even a viewport. However, it kept feeling like it's stuck. Not stuck as in "this is going slow", but stuck as in "not moving at all".

I don't know how to measure the progress of UI development. Usually, with backend development (which is mostly what I do), I can see the project moving (even if slowly), but here I don't. It reminds me of the stage in your code when you're still guessing yourself on the way you planned out your program, on your algorithms, on your assumptions and supposed requirements. You keep going back to the drawing board, unsure of your project.

Last night I sat once again to try and sketch things out. I decided this time to sketch the usability of the UI instead of the appearance. The difference for me means I think of what would happen instead of how it would look like. I started sketching arrows and write small comments (double click, single click, tab). Things got better, I finally felt like I know my goals, at least the ones in the beginning.

I sat down, wrote a viewport, added a tab panel, added a grid on the first panel and was even able to open and move to a new tab on double click of a record. Pretty cool. At first I was excited about this, but then dejected. Was this progress? How would I know if I made progress just now, or got stuck on something irrelevant?

I assume working on the parts that I already know by heart how to do would be a waste, which is the backend. I should focus on the frontend, that is clear. But should I focus on the panels? On the tabs? On fetching the data? Should I be picking at the grid? Should I be picking at the functionality? What about the layout? Should I be learning how to do everything in ExtJS first? Maybe strengthen a bit on the Javascript side?

I'm feeling quite confused, something that although happens often in life, doesn't occur to me often on the technical side. There are so many things to pick from and I don't know which one is correct. I reckon that is why I keep feeling like I'm making no progress. Maybe I should sketch down all the possible components and then try to assemble a puzzle out of them and then write the functionality. I don't know.

Does anyone have any advice on this?

08 Mar 2010 1:00pm GMT

Major release of Dancer finally out. You can read about it on Alexis' blog.

Basically this version provides a lot of improvements to Dancer:

• 
  
• Alexis Sukrieh worked hard on refactoring and optimizing.
• 
  
• David Precious did a lot of awesome work on documentation: we now have a cookbook and a deployment manual for various situations you might encounter. He also added a session backend with clean recursive dumps.
• 
  
• I added the route caching, which shows a significant speed improvement!
• 
  
• We have a mailing list
• 
  
• We have a new main domain (perldancer.org) and it's the first project to use a perlprojects.net subdomain: dancer.perlprojects.net.
• 
  

I want to thank Sukriah and David for all their work and being great guys to work with.

Bust a move!

08 Mar 2010 9:49am GMT

It's never great to post an "end-February" report a week into March, but that's how things are going lately. I've been busy with family and work obligations that have meant less CT2.0 hacking. I'm sorry this is coming late, but I hope I will give anyone interested a sense of where things stand.

I should note that the original deadline for finishing CT2.0 was March 1 and clearly we're not there yet. I've discussed the situation with Robert and Ask at the Perl NOC, and they've been willing to extend the deadline for cutting off CT1.0 for a while longer. Thank you, Robert and Ask, for your understanding!

Progress in the last couple weeks:

• I did some alpha testing of the CT2.0 Metabase hosted on Amazon. Based on that, I revised yet again (sigh) the user registration/credentials approach to minimize the hassle for old and new registrants.
• Florian Ragwitz wrote a Catalyst app to help distribute new credentials files to legacy CT1.0 users. I haven't deployed it yet (since I now need to regenerate all the credentials), but greatly appreciate his quick turnaround.
• I implemented the Metabase search capabilities that Barbie will need to update the CPAN Testers statistics database. This will be based on the excellent SQL::Abstract approach to WHERE clause construction.
• I wrote several helper modules to simplify configuration of a CPAN Testers metabase in preparation for deployment. The first of these has already been uploaded to CPAN: Net::Amazon::Config
• I finalized the "version 0″ API for the Metabase web service and revised the interface between the Catalyst app and the Metabase to reflect the latest changes to the library.

In the last several weeks, members the #catalyst and #moose IRC channels were very, very helpful and patiently answered my many stupid questions. Thank you in particular to confound, perigrin, rafl, rjbs, stevan and t0m.

Coming up:

• Deploy all my new code onto the server for "beta" testing
• Release all the code to CPAN that people will need to configure their clients for beta testing
• Regenerate user profiles and deploy rafl's app to distribute them to legacy users
• Whip the server into production shape (e.g. proper boot scripts to auto-start the CT2.0 apps on restart)
• Get back to work on legacy report migration

It's at the point now where I suspect the "hard thinking" part is pretty much done and it's a lot of grotty but straightforward tasks to go. Hopefully, beta testing won't reveal any major issues and the end of March update will be focused on planning on orderly transition from CT1.0 to CT2.0.

08 Mar 2010 4:59am GMT

an impressive post like that one a week ago won't come again so fast. Last days I added Pawel Murias and Gabor and mentioned pixie, but the main part is done here. The timeline (structured and with 33 items) is now also mostly done. You can also see that we exceeded the zenith of edits. The number of articles touched in last 14 days is now sinking rapidly. it So whats next?

Well renee needs the next perl 6 article, which I want deliver this week. All the experience I collected writing the 200+ changes in last 2 weeks will go into that. But my next goal for the TPF wiki will be the translation of my perl 6 tut and the release of Kephra 0.4.3, which is some weeks overdue.

08 Mar 2010 1:21am GMT

The last week or so, a lot of the work on Strawberry Perl has gone to making sure that what I'm calling the "gcc4" toolchains from the mingw64 project will work to build Strawberry Perl.

And it's looking like that will happen, as the two toolchains (a 32-bit one and a 64-bit one) will work, at least as far as building what's been called "Vanilla Perl" in the past is concerned.

There are three XS modules in current versions of Strawberry that have issues with 64-bitness that I know of so far. (Compress::unLZMA, Math::Pari, and Crypt::OpenSSL.) I may find out about more as I attempt to build Strawberry Perl.

KMX has built new versions of the libraries that some XS modules rely on (postgresql, mysql, libgd, libtiff, libjpeg, etc.) that should not have name conflicts with other DLL's on the system, and the 32-bit versions will work with both the gcc3 and gcc4 toolchains.

I'm attempting a 32-bit build tonight of Strawberry Perl with 5.11.5, gcc4, and the new libraries. After that, I'll attempt to do the same with 5.10.1.

Soon will also be a 64-bit test with 5.11.5 and the 64-bit "new libraries".

As I've previously stated, 5.10.x versions of Strawberry will be 32-bit only, and will stay on the current "gcc3" toolchain. They WILL use the new libraries - which will solve DLL problems that have sometimes occured when other programs have DLL's with the same name.

I've said in the past that we'd have a 64-bit version of Strawberry available for April. That will be both true and false, for I won't be able to release a non-beta version of 5.12.0 before April 30th (for that to happen, I'd have to have the perl 5.12.0 tarball available to build upon before April 2nd, and I don't think that'll happen, with apologies to the perl5-porters.)

There will be a 64-bit beta available soon - most likely by the end of this month. While I'm currently testing 64-bitness with 5.11.5, 5.11.6 will probably be available in 2 weeks, and I'll release the beta soon after that. There will be NO non-beta 5.11.x versions of Strawberry Perl - as those perl versions are practically betas themselves.

The intent is to have 3 or 4 beta test versions of Strawberry Perl for the first beta towards the April release:

1. 
   
2. A 32-bit 5.10.1.2.
   
3. A 32-bit 5.11.6.0.
   
4. A 64-bit 5.11.6.0.
   
5. A 32-bit 5.10.1.2 with the Professional additions may or may not happen.
   

Watch for the 5.10.1.2 beta to come out around St. Patrick's Day, with 5.11.6.0 betas a week or so afterwards.

07 Mar 2010 7:22pm GMT

I've been continuing to work on adding improved logging capabilities to Dist::Zilla. They're not going to be terribly complex, but I'm trying to make sure that they're easy to use, and that they lend themselves to sane default output and useful debugging output. Yesterday, I realized that these lines were really annoying:

[DZ] Dist::Zilla::PluginBundle::RJBS/Dist::Zilla::PluginBundle::Git/Dist::Zilla::Plugin::Git::Push initialized

I went through and fixed all the bundles I could to produce simpler (but still useful) names and then used each plugin's logger for logging initialization. The result is now more like this:

[@RJBS/@Git/Push] online, Dist::Zilla::Plugin::Git::Push v1.234567

That's also been demoted to debug output, so there's much less noise by default. Releasing Dist::Zilla this morning looked like this:

~/code/cs/Dist-Zilla$ dzil release
reading configuration using Dist::Zilla::Config::Finder
[DZ] beginning to build Dist-Zilla
[DZ] guessing dist's main_module is lib/Dist/Zilla.pm
[DZ] extracting distribution abstract from lib/Dist/Zilla.pm
[@RJBS/Classic/ExtraTests] rewriting release test xt/release/pod-coverage.t
[@RJBS/Classic/ExtraTests] rewriting release test xt/release/pod-syntax.t
[DZ] writing Dist-Zilla in Dist-Zilla-1.100660
[DZ] writing archive to Dist-Zilla-1.100660.tar.gz
[@RJBS/@Git/Check] branch master is in a clean state
[@RJBS/Classic/UploadToCPAN] registering upload with PAUSE web server
[@RJBS/Classic/UploadToCPAN] POSTing upload for Dist-Zilla-1.100660.tar.gz
[@RJBS/Classic/UploadToCPAN] PAUSE add message sent ok [200]

Not bad!

I also made some improvements to Pod::Weaver and the PodWeaver plugin so that it can log its work as it goes. This will produce output like:

[@RJBS/PodWeaver] [Name] added name/abstract section to lib/Dist/Zilla.pm

I'm starting to wonder if Dist::Zilla isn't the project where I'll end up wanting more fine-grained control than "normal logging" or "debugging logging," but I think things will go as this issue always has: I'll put it off until everything else is done and then realize I don't care about it.

07 Mar 2010 2:22pm GMT

• we touched 101 articles in last 2 weeks
• currently I'm writing on the Perl 6 timeline
• I saved the content of ponie's page into its's wiki page because it might will soon disappear

04 Mar 2010 11:35pm GMT

Oops. I think excessive distractedness just made me miss a 10-day interval, thereby falling off the Iron Man challenge. Oh well. [Sidenote: Is there a way to easily check one's Ironman status?]

Lately, I've been feeling a bit like the snowplows shuffling snow around outside my office window. I have a lot of things I want to blog about, but I've been pushing them ahead of me. That's exactly what the Iron Man thing is supposed to counter. Guess procrastination won out in this case.

So, how much blog would a masak write if a masak could write blog? Here's a list off the top of my head of the things I've been thinking of blogging about:

• The presentation I'm writing for the Open Source Days in Copenhagen this weekend. I'm really excited about it.
• The "7 Wonders of the Ancient Grammar Engine" series. I've started on it, and I like what I have so far.
• More E03 stuff. Guess that's what I didn't blog about in the past 11 days.
• snarkyboojum++ and I have started toying with writing a time-travelling debugger. Yes, really.
• A lot of things are happening in Rakudo-land this month, after the successful ng merge. colomon++ especially shines like a bright star right now, bringing lots of tests back online each day.
• I'd like to get enums in Rakudo before this month's release. I've already done anon enums. Next up: named enums.
• I've been starting to think seriously about u4x lately. Partly because of other people's questions, partly because it seems the the time is ripe to start it.
• However, every time I have that thought I realize that the book is more important at the moment. I have some unimplemented ideas there as well.
• GGE is very near being PGE-compliant. Just a few finishing touches are needed. This will likely usher in a new era in Perl 6 grammars and parsing.

Will I have time in the near future to expand these one-liners into full-fledged blog posts? Only time will tell.

02 Mar 2010 11:15am GMT

The Perl 6 design team met by phone on 24 February 2010. Larry, Allison, Patrick, and chromatic attended.

Larry:

• my work last week was almost entirely responsive to various discussions on irc and p6l, even when it doesn't seem like it
• clarified that LEAVE-style phasers do not trip till after an exception is handled (and not resumed)
• the implementation of take is specifically before unwinding even if implemented with a control exception
• simplified series operator by moving generator function to the left side (any function on right side will now be a limiting conditional)
• a * is no longer required to intuit the series on the left; the absence of generator before the ... operator is sufficient
• first argument on the right of ... is now always a limiter argument
• for convenience and consistency, added a new ...^ form to exclude a literal limiter from the generated series
• unlike ranges, however, there is no leading exclusion ^... or ^...^
• series is a list associative list infix, and each ... pays attention only the portion of the list immediately to its left (plus the limit from the right)
• an "impossible" limit can terminate a monotonic intuited series even if the limit can never match exactly
• variables now default to a type of Any, and must explicitly declare Mu or Junction type to hold junctions
• this is to reduce pressure to duplicate many functions like == with Mu arguments; most of our failure values should be derived from Any in any case
• a Mu result is more indicative of a major malfunction now, and is caught at first assignment to an Any variable
• Instant/Duration types are biased away from Num and towards Rat/FatRat semantics
• Instant is now completely opaque; we no longer pretend to be the same as TAI, numerically speaking
• Instants are now considered a more basic type than epochs, which are just particular named instants
• all culturally aware time can be based on calculations involving instants and durations
• list associative operators now treat non-matching op names as non-associative rather than right-associative, forcing parens
• Whatever semantics now autocurry any prefix, postfix, or infix operator that doesn't explicitly declare that it handles whateverness itself
• WhateverCode objects now take a signature to keep clear how many args are not yet curried
• so *+* is now more like WhateverCode:($x,$y)
• autocurrying is still transitive so multiple ops can curry themselves around a *
• added semilists as Slicel type to go with Parcel
• this allows us to bind @array[1,2,3] differently from @array[1,2,3;4,5,6], for instance
• the Matcher type now excludes Bool arguments to prevent accidental binding to outer $_ when closure is needed
• when and ~~ will now warn of always/never matching on direct use of True or False names as matcher
• STD generalizes \w lookahead to all twigils now
• STD now treats non-matching list associatives as non-associative
• things like 1 min 2 max 3 are now illegal, and require parenthesization for clarity
• STD now treat invocant colon as just a comma variant so it does not fall afoul of the list associativity change
• CORE now recognizes the TrigBase enumeration

Patrick:

• first release of the new branch of Rakudo last week
• passing ~25,000 tests at the release
• thanks to optimizations from chromatic, Jonathan, and Vasily, Rakudo has a lot of speed improvements
• in particular, it can run those tests in under 10 minutes, non-parallel, depending on your hardware
• older releases took 25 minutes and more
• the regex tests will slow things down
• ultimately, we're seeing a big speed improvement over the past releases
• cleaned up lists and slices, now they work pretty well
• worked with Solomon Foster and others to speed up trig operations
• fixed a bug related to lexicals declared in classes
• fixed the long-standing and often recurring problem with curlies ending a line/statement causing the next statement to be a statement modifier
• easy to fix in the new grammar
• that was nice
• made an initial implementation of the sort method
• it's very short, because Parrot provides one
• there are a few bugs in Rakudo there still, but I'll get them
• planning for the Copenhagen hackathon on March 5 - 9
• Jonathan and I have been updating the Rakudo roadmap
• will check that in in the next couple of hours
• so far, every time we review it, we surprise ourselves at how much we've accomplished
• we're meeting all of the top priority goals without making any heroic efforts
• we'll put those goals in as well as timelines
• most of the major tasks from previous roadmaps have happened

Allison:

• working on Python this week
• attended Python VM summit, Python language summit, and PyCon
• Parrot's on good track to support what Python needs
• useful to make community connections
• when I reviewed Pynie, I was surprised to see how close it is to supporting the whole Python syntax
• some of those features are big, like objects
• but we should support them soon
• Debian packages delayed by the absence of a sponsor
• they should go into Debian soon though
• I put in a request for feature-freeze exception for Ubuntu 10.4
• Parrot 2.0 should go in
• haven't made any commits to the PCC branch
• that'll be a top priority for next week

c:

• fixed a Parrot GC bug for last week's Rakudo release
• made some optimizations in Rakudo and Parrot
• helped Jonathan find a few more
• fixed a long-standing math MMD bug
• still working on HLL subclassing; more tricky than you think
• may be some conflicting design goals about vtable overriding and MMD

Allison:

• Patrick, do we need an explicit deprecation for old PGE and NQP?

Patrick:

• I think Will already added one for NQP
• we can add one for PGE if we need
• they don't necessarily have to disappear at the next release
• but no one's planning to maintain them

Allison:

• no reason not to put in the notice now
• we don't have to remove them at the earliest possible date

02 Mar 2010 5:12am GMT

The wiki madness continues, I did 2,3 People stubs(Will Coleda, Gabor) and a lot stubs around Parrot: PCT, NQP, Blizkost, PIR, Parrot compiler, PGE but that will slow down. There are some community related things missing like conferences, hackathons and so on and maybe a perl 6 timeline but what I want to show today, are articles which became well formated good content.

• Documentation
• Parrot
• Sprixel

Sprixel was done its coauthor Martin.

02 Mar 2010 1:15am GMT

Now its's the third time I'm rambling about the recent TPF wiki overhaul. Beside the fresh cleaned frontpages that just containes links to the 5 most important pages and aggregation of recent recent blog posts (was there before), many new pages were created. before we had 3 or 4 pages about wiki contributers now we have:

• Larry Wall
• chromatic
• Damian Conway
• Allison Randal
• Audrey Tang
• Daniel Ruoso
• Jonathan Worthington
• Flávio Soibelmann Glock
• Patrick Michaud
• Moritz Lenz
• Jonathan Leto
• Stephen Weeks
• Carl Mäsak

Kudos to lot of these people who helped me to write their article. Masak is last because the November bug is still there. :) Then we had an article about KP6, SMOP , Rakudo and Parrot. Now we have:

• Implementations
• Rakudo
• Pugs
• viv
• vill
• mildew
• Sprixel
• Elf
• Perlito
• KindaPerl6
• v6
• Historical Implementations

Special thanks here to chromatic++. Yes, yes almost all of them are stubs, but i add constantly. then I also added:

• Specification
• Synopses
• STD.pm
• Test Suite
• Rakudo Star
• Open_source_Perl_6_book
• Whats_up?
• FAQ

Like said, many of them are very short, but sometimes its only necessary to give form like a crystalization point.

25 Feb 2010 11:46pm GMT

The Perl 6 design team met by phone on 17 February 2010. Larry, Allison, Patrick, and chromatic attended.

Larry:

• much work clarifying relationship of parcels to everything else (<a b>, assignment, arguments, captures, parameters, signatures, gather/take, and loop returns)
• we now list all scope declarators in one spot
• conjectured some ideas on how to handle the allomorphism of literals more dwimmily
• had already specced some of this behavior for literals found inside qw angles.
• literals that exceed a Rat64's denominator automatically keep the string form around for coercion to other types
• clarified that anon declarator allows a name but simply doesn't install it in the symbol table
• respecced the trig functions to use a pragma to imported fast curried functions
• still uses enum second argument for the general case (rakudo is still stuck on slow strings there)
• on iterators, renamed .getobj to .getarg since arguments are the typical positional/slicey usage
• signatures are never bound against parcels anymore, only against captures
• we now use "argument" as a technical term meaning either a real parcel or an object that can be used independent of context as an argument
• anything that would stay discrete when bound to a positional, basically
• return, take, and loop return objects are also arguments in that sense
• they all return either a parcel or anything that can stand on its own as an argument
• STD now adds a shortname alias on adverbialized names, ignores collisions on the shortname for now, which is okay for multis
• STD now complains about longname (adverbialized) collisions
• STD no longer carps about duplicate anonymous routine declarations
• made the undeclared type message the same for parameters as for other declarations
• clarify the error message about anonymous variables
• no longer report a $) variable error where ) is the $*GOAL
• add WHAT etc. to list of functions that require an argument

Allison:

• working on two HLL implementations
• one is Pynie, the other is Camle
• nothing to do with Caml or ML
• I've noticed huge improvements in NQP-rx from the previous NQP
• can't say which feature improvements make the most difference, but I'll migrate Pynie pretty soon to take advantage of the new version
• continuing to shepherd Debian and Ubuntu packages

Patrick:

• essentially all I did was unify things
• previously it had been two or three tools
• it's just one

Allison:

• even the syntax seems more regular

Patrick:

• there are more pieces available in NQP-rx
• Rakudo's -ng is now master
• the old master is now -alpha
• we took a big hit on spectests, but they seem to be coming back quickly
• 5000 tests pass on trunk now
• we have 16k or 17k we haven't re-enabled; they make the spectest slower
• Jonathan thinks we may pass 25,000 tests now
• that's great, considering where we were a week ago
• I redid Rakudo's container, value, and assignment module
• previously variables held values directly
• now they contain reference PMCs
• that cleaned up many things
• we use more PMCs, but now we don't clone and copy as much
• we move references around more
• seems closer to how Perl 6 handles things
• was much easier than I expected
• updated the NQP-rx regex engine and built in constant types
• handles Unicode character names
• reclaims plenty of tests
• answered lots of questions for people adding things into Rakudo
• prioritizing other people writing code over writing code
• increases our developer pool; seems to be working well
• new release of Rakudo planned for tomorrow
• don't know how many tests we'll pass, but it should go well
• plan to put in a few things like sort and grammars over the next week
• then I'll review the RT queue to find bugs and (hopefully) closeable bugs

c:

• working on GC tuning
• also working on String PMC tuning
• working on built-in types and their behavior as classes and parent classes
• the multidispatch bugs in particular I hope to solve

25 Feb 2010 12:27am GMT

He was a major writer in the TPF wiki. At least half of the todays visible content is his and ruosos typework, even if I did spit a lot out these days. Because these efforts, other than his 1000$ donation for November, are largely unrecognized I solute him here. That doesn't mean I brag with my stuff too.

He gave the new frontpage the final polish.

Currently I am focussed on entering all major people and implementations into the TPF wiki. So check please if I'm writing something wrong about you. Yes and chromatic, Allison and Damian are still missing.

23 Feb 2010 2:53am GMT

together with mberends++ we revamped the fronpage of the TPF wiki, sorted links, pages, deleted spam and added some minor content. There are still some old links left but camelia smiles now.

20 Feb 2010 2:26am GMT

My signature improvements Hague Grant is pretty much wrapped up. I wrote a couple of posts already about the new signature binder and also about signature introspection. In this post I want to talk about some of the other cool stuff I've been working on as part of it.

First, a little background. When you make a call in Perl 6, the arguments are packaged up into a data structure called a capture. A capture contains an arrayish part (for positional parameters) and a hashish part (for smok^Wnamed parameters). The thing you're calling has a signature, which essentially describes where we want the data from a capture to end up. The signature binder is the chunk of code that takes a capture and a signature as inputs, and maps things in the capture to - most of the time, anyway - variables in the lexpad, according to the names given in the signature.

Where things get interesting is that if you take a parameter and coerce it to a Capture, then you can bind that too against a signature. And it so turns out that Perl 6 allows you to write a signature within another signature just for this very purpose. Let's take a look.

multi quicksort([$pivot, *@values]) {
my @before = @values.grep({ $^n < $pivot });
my @after = @values.grep({ $^n >= $pivot });
(quicksort(@before), $pivot, quicksort(@after))
}
multi quicksort( [] ) { () }


Here, instead of writing an array in the signature, we use [...] to specify we want a sub-signature. The binder takes the incoming array and coerces it into a Capture, which essentially flattens it out. We then bind the sub-signature against it, which puts the first item in the incoming array into $pivot and the rest into @values. We then just partition the values and recurse.

The second multi candidate has a nested empty signature, which binds only if the capture is empty. Thus when we have an empty list, we end up there, since the first candidate requires at least one item to bind to $pivot. Multi-dispatch is smart enough to know about sub-signatures and treat them like constraints, which means that you can now use multi-dispatch to distinguish between the deeper structure of your incoming parameters. So, to try it out...

my @unsorted = 1, 9, 28, 3, -9, 10;
my @sorted = quicksort(@unsorted);
say @sorted.perl; # [-9, 1, 3, 9, 10, 28]


It's not just for lists either. An incoming hash can be unpacked as if it had named parameters; for that write the nested signature in (...) rather than [...] (we could have use (...) above too, but [...] implies we expect to be passed a Positional). For any other object, we coerce to a capture by looking at all of the public attributes (things declared has $.foo) up the class hierarchy and making those available as named parameters. Here's an example.

class TreeNode { has $.left; has $.right; }
sub unpack(TreeNode $node (:$left, :$right)) {
say "Node has L: $left, R: $right";
}
unpack(TreeNode.new(left => 42, right => 99));


This outputs:

Node has L: 42, R: 99


You can probably imagine that a multi and some constraints on the branches gives you some interesting possibilities in writing tree transversals. Also fun is that you can also unpack return values. When you write things like:

my ($a, $b) = foo();


Then you get list assignment. No surprises there. What maybe will surprise you a bit is that Perl 6 actually parses a signature after the my, not just a list of variables. There's a few reasons for that, not least that you can put different type constraints on the variables too. I've referred to signature binding a lot, and it turns out that if instead of writing the assignment operator you write the binding operator, you get signature binding semantics. Which means...you can do unpacks on return values too. So assuming the same TreeNode class:

sub foo() {
return TreeNode.new(left => 'lol', right => 'rofl');
}
my ($node (:$left, :$right)) := foo();
say "Node has L: $left, R: $right";


This, as you might have guessed, outputs:

Node has L: lol, R: rofl


Note that if you didn't need the $node, you could just omit it (put keep the things that follow nested in another level of parentheses). This works with some built-in classes too, by the way.

It works for some built-in types with accessors too:

sub frac() { return 2/3; }
my ((:$numerator, :$denominator)) := frac();
say "$numerator, $denominator";


Have fun, be creative, submit bugs. :-)

20 Feb 2010 12:21am GMT

I was a bit optimistic with time. So what else is new.

Anyway, various distractions such as $WORK kept interrupting my translation of E03 today, but now I've finally got it all ported to modern Rakudo-Perl 6. Here it is. Two biggish things remain:

1. I haven't tried running the code. I'm pretty sure it won't run on the new Rakudo master yet. But I've written it with alpha (the old development branch) in mind, and it should run there.
2. I haven't looked at SF's E03 translation yet. (Yes, I'm linking to a blog entry which I haven't read yet.) I saw that the title of his post is "first stab", which makes me feel a little better.

Here are a few random comments about the code.

• The original code in E03 is the strangest example code I've ever read. All due props to Damian, but... quod? A program whose task it is to "locate a particular data file in one or more directories, read the first four lines of each such file, report and update their information, and write them back to disk"? Come again? (My suspicion is that this was the program arrived at by simply cramming in as much of A03 as possible into the same piece of code, but nevermind.)
• A few nice things are still missing from Rakudo. The s{}{} syntax is the last still unimplemented request in an old blog post of mine. It'd be pretty nice to have. I think we'll have it soon, thanks to ng.
• You can't open files rw in Rakudo. Then again, you can't seek or truncate them either, so I guess opening them rw wouldn't make much sense anyway. Due to this, I had to work around a central part in the program, where filehandles were stored in a hashtable, to be read from in one subroutine and written to in the next one. Had to store the filenames instead, and re-open the files for writing.
• We don't have the :r and :w filetests yet, but :e and (strangely enough) :s are implemented. Someone with tuits should really do :r and :w.
• A fair amount of the updates in E03 are out-of-date. I guess operators and related things are an area where Perl 6 really has evolved a lot in the past few years, or even in the past year. As I speak, the exact semantics for infix<...> is still being discussed on #perl6.
• When I say things have changed around a lot, I should really mention that I say that with a sort of relief of the sort that means "gee, I'm glad we didn't stick to what we thought we wanted back then!". Things have, by and large, improved greatly. I actually hope to write more about that.

This exigesis-modernising is kinda fun! Now I'm eager to go read what SF++ has been up to.

19 Feb 2010 11:30pm GMT

On behalf of the Rakudo development team, I'm pleased to announce the
February 2010 development release of Rakudo Perl #26 "Amsterdam".
Rakudo is an implementation of Perl 6 on the Parrot Virtual Machine
(see http://www.parrot.org). The tarball for the February 2010 release
is available from http://github.com/rakudo/rakudo/downloads .

Rakudo Perl follows a monthly release cycle, with each release named
after a Perl Mongers group. The February 2010 release is code named
"Amsterdam" for the largest chapter of the Dutch Perl Mongers. Perl
development enjoys considerable support from the Netherlands, with
donations from NLNet, and hosting of the feather machines and several
important Perl 6 web domains and sites.

This release is the first release based on the new branch of
Rakudo development begun in October 2009. The branch refactors
the grammar, object metamodel, and a number of other key features
to improve compatibility with the Perl 6 specification and give us
a more solid foundation to build on. Indeed, in many ways the
development of this new branch has driven important changes to the
specification in the areas of lists, iterators, slices, and much
more.

However, this release contains a number of significant regressions
from previous compiler releases. We expect to have full functionality
restored in this branch in the next couple of weeks. For those
looking to explore a wide variety of Perl 6 features or who have
applications developed using previous releases of Rakudo, you may
wish to continue to use the January 2010 (#25, "Minneapolis")
release.

This release of Rakudo requires Parrot 2.1.0. One must still
perform "make install" in the Rakudo directory before the "perl6"
executable will run anywhere other than the Rakudo build directory.
For the latest information on building and using Rakudo Perl, see the
README file section titled "Building and invoking Rakudo".

Some of the specific changes and improvements occuring with this
release include:

* Now using nqp-rx for parsing and actions

* Grammar is much closer to STD in many aspects, and makes use of
protoregexes

* Closures and lexical/contextual variable declarations in regexes work

* Laziness is implemented

* All class and role construction is handled through the meta-model

The Perl 6 language specification is still in flux. Please take note of the
following changes, which might affect your existing programs. In the next
release of Rakudo, the deprecated features will likely be gone.

* The root of the object hierarchy has been changed from 'Object' to 'Mu'.
The type 'Object' goes away.

* The term 'undef' is gone. You can replace it with other constructs,
depending on context:
- 'Nil' is undefined in item context, and the empty list in list context
- 'Mu' is the most general undefined value which does not flatten in
list
context
- as a smart matching target, you can replace '$obj ~~ undef'
by '$obj ~~ *.notdef'

* Builtin classes will derive from 'Cool' (which itself derives from 'Any').
Most of the builtin methods on these classes will be defined in the
'Cool' class instead of 'Any'. See Synopsis 2 for more details.

* Starting with the this release, release identifiers are given
as "YYYY.MM" instead of "YYYY-MM" (dot instead of hyphen).
This is intended to simplify building and packaging for other
distribution systems.

The development team thanks all of our contributors and sponsors for
making Rakudo Perl possible. If you would like to contribute,
see http://rakudo.org/how-to-help , ask on the perl6-compiler@perl.org
mailing list, or ask on IRC #perl6 on freenode.

The next release of Rakudo (#27) is scheduled for March 18, 2010.
A list of the other planned release dates and codenames for 2010 is
available in the "docs/release_guide.pod" file. In general, Rakudo
development releases are scheduled to occur two days after each
Parrot monthly release. Parrot releases the third Tuesday of each month.

Have fun!

[1] http://www.frozen-perl.org/
[2] http://use.perl.org/~pmichaud/journal/39779
[3] http://use.perl.org/~pmichaud/journal/39874

19 Feb 2010 9:58am GMT

On behalf of the Rakudo development team, I'm pleased to announce the
February 2010 development release of Rakudo Perl #26 "Amsterdam".
Rakudo is an implementation of Perl 6 on the Parrot Virtual Machine
(see http://www.parrot.org). The tarball for the February 2010 release
is available from http://github.com/rakudo/rakudo/downloads .

Rakudo Perl follows a monthly release cycle, with each release named
after a Perl Mongers group. The February 2010 release is code named
"Amsterdam" for the largest chapter of the Dutch Perl Mongers. Perl
development enjoys considerable support from the Netherlands, with
donations from NLNet, and hosting of the feather machines and several
important Perl 6 web domains and sites.

This release is the first release based on the new branch of
Rakudo development begun in October 2009. The branch refactors
the grammar, object metamodel, and a number of other key features
to improve compatibility with the Perl 6 specification and give us
a more solid foundation to build on. Indeed, in many ways the
development of this new branch has driven important changes to the
specification in the areas of lists, iterators, slices, and much
more.

However, this release contains a number of significant regressions
from previous compiler releases. We expect to have full functionality
restored in this branch in the next couple of weeks. For those
looking to explore a wide variety of Perl 6 features or who have
applications developed using previous releases of Rakudo, you may
wish to continue to use the January 2010 (#25, "Minneapolis")
release.

This release of Rakudo requires Parrot 2.1.0. One must still
perform "make install" in the Rakudo directory before the "perl6"
executable will run anywhere other than the Rakudo build directory.
For the latest information on building and using Rakudo Perl, see the
README file section titled "Building and invoking Rakudo".

Some of the specific changes and improvements occuring with this
release include:

* Now using nqp-rx for parsing and actions

* Grammar is much closer to STD in many aspects, and makes use of
protoregexes

* Closures and lexical/contextual variable declarations in regexes work

* Laziness is implemented

* All class and role construction is handled through the meta-model

The Perl 6 language specification is still in flux. Please take note of
the
following changes, which might affect your existing programs. In the next
release of Rakudo, the deprecated features will likely be gone.

* The root of the object hierarchy has been changed from 'Object' to 'Mu'.
The type 'Object' goes away.

* The term 'undef' is gone. You can replace it with other constructs,
depending on context:
- 'Nil' is undefined in item context, and the empty list in list
context
- 'Mu' is the most general undefined value which does not flatten in
list
context
- as a smart matching target, you can replace '$obj ~~ undef'
by '$obj ~~ *.notdef'

* Builtin classes will derive from 'Cool' (which itself derives from
'Any').
Most of the builtin methods on these classes will be defined in the
'Cool' class instead of 'Any'. See Synopsis 2 for more details.

* Starting with the this release, release identifiers are given
as "YYYY.MM" instead of "YYYY-MM" (dot instead of hyphen).
This is intended to simplify building and packaging for other
distribution systems.

The development team thanks all of our contributors and sponsors for
making Rakudo Perl possible. If you would like to contribute,
see http://rakudo.org/how-to-help , ask on the perl6-compiler@perl.org
mailing list, or ask on IRC #perl6 on freenode.

The next release of Rakudo (#27) is scheduled for March 18, 2010.
A list of the other planned release dates and codenames for 2010 is
available in the "docs/release_guide.pod" file. In general, Rakudo
development releases are scheduled to occur two days after each
Parrot monthly release. Parrot releases the third Tuesday of each month.

Have fun!

[1] http://www.frozen-perl.org/
[2] http://use.perl.org/~pmichaud/journal/39779
[3] http://use.perl.org/~pmichaud/journal/39874

19 Feb 2010 9:57am GMT