11 Mar 2010
Planet Turbogears
Plumbling Life's Depths: Infinite Deployment Docs and OpenID Issues
Have spent rather a lot of time trying to wrestle the deployment docs into shape for the TurboGears 2.1 release. Even with all that work, only the "standard deployment pattern" is really in "good" shape. Alternative servers, alternative deployment styles, etceteras are all still needing lots of love. I am, however, I think, finished with that section for a while unless someone else chips in to help.
Also ran into one of those "argh" inducing issues this evening. A user was trying to follow through the "Using who.ini" discussion on his way to doing OpenID with TurboGears and ran into a weird, subtle bug. Turns out, when you disable TurboGears authentication repoze.what "fails open" as far as much TurboGears code is concerned. The predicates just all silently start evaluating to True. The @require() decorator thankfully does the right thing, but users who do:
tg.predicates.has_permission('manage')
Now always get a True value (the predicate object), so if you're using that to control display of information... oops. Not likely to have bitten too many users, but it just seems we should have something raising errors on _nonzero_() if the predicate's aren't "booleanized()".
In other news, given that someone else is trying to get OpenID working, I suppose I should look at the branch of repoze.openid. Upstream wants changes, but I haven't yet figured out what's supposed to happen wrt registration to make those changes work. Repoze just doesn't seem to have a registration story AFAICT.
11 Mar 2010 10:09am GMT
09 Mar 2010
Planet Turbogears
Michael Pedersen: JSOn vs XML vs YAML, and Python Parsing
EDIT: I added YAML testing data to the results. I'm not posting my original files, as I don't have a license to be converting this game, and don't want to trip any legal issues. Suffice to say that the names in the file give it away.
EDIT: I added ElementTree testing data to the results.
I've begun working on translating a board game into a computer game. In order to do so, I need to be able to represent the state of the game and its many (many) tokens. In addition, I need to be able to store and load that state quickly. Finally, I need that state to be something that I can represent with a single string.
Storing the state presents a simple hierarchical structure: Multiple decks of cards are shuffled, and the shuffled state must be saved. This allows for a web front end to the game. The cards are just one aspect: I have to store information about what cards the player has, what state the cards are in, where the player's token is, and what feels like about 500 pieces of information (no joke: I scanned 494 images for this game).
So, how do I represent this data? How do I make it so that a web server can load and save the state quickly? XML is good for hierarchical data, as are JSON and YAML. But which one is right this time?
I needed to test it out. So, I wrote three equivalent files, one in XML, one in JSON, one in YAML. I then ran the timeit module against those files, using the following commands:
python -m timeit -n 100 -s 'from xml.dom.minidom import parse' 'd=parse("allboard.xml")'
python -m timeit -n 100 -s 'from xml.etree.ElementTree import parse' 'd=parse("allboard.xml")'
python -m timeit -n 100 -s 'import simplejson' 'd=simplejson.load(open("allboard.json"))'
python -m timeit -n 100 -s 'import yaml; from yaml import CLoader as Loader' ' d=yaml.load(open("allboard.yaml"), Loader=Loader)'
The results were simply staggeringly different.
XML (minidom) took 15.3ms per run. XML (ElementTree) took 19.6ms per run. JSON took 250usec per run. YAML took 10.1ms per run when using the CLoader. Without the CLoader, it took 115ms per run.
JSON wins this one hands down.
Is it possible I chose poorly? Of course. There are many XML parsers for Python, just as there are many JSON parsers. I wanted pure Python here, so as to maximize portability. I chose these modules since they should be completely standard. If I'm wrong, tell me, and I'll revisit this.
09 Mar 2010 5:36am GMT
08 Mar 2010
Planet Turbogears
Plumbling Life's Depths: RunSnakeRun gets a basic source-code-view
There was a request from one of the RedHat guys to add a source-code view to RunSnakeRun so that users can just flip over to see the code which is running slow when they are exploring a profile. wx.py makes that pretty easy, so bzr head now supports it.
Want to maybe add a call-tree view. A memory-profiling view would be nice too, but I don't have a recommended memory profiler at which to point people. Without that it's a bit of a toss-up as to what memory-profile format to support.
08 Mar 2010 2:14am GMT
02 Mar 2010
Planet Turbogears
Michael Pedersen: Announcing the Availability of tgext.menu v1.0b1
It's been a lot longer in coming than it should have been, maybe, but it's here finally: The menuing extension for TurboGears. This extension gives TurboGears developers the ability to stop looking for where their controllers are set, how to integrate new controllers into their navigation menus, etc.
All of that goes away. Now you can just use simple functions and decorators to generate your menus for you. As it also uses jdMenu, you get dynamic menus that are very suitable for a web application.
If you're a TurboGears developer, you owe it to yourself to check it out today.
02 Mar 2010 6:24am GMT
01 Mar 2010
Planet Turbogears
Christopher Perkins: Pycon 2010 Dump… Sorta
I decided this year I am going to forego my Pycon dump, and focus on the news that matters most to my readership. There are plenty of reviews of talks, and I went to a bunch of good ones, had a great time at wsgi house, blah, blah, blah. What _you_ probably care about is what's cutting edge in TurboGears, and the sprints are where it's at.
We had 12 sprinters on day 1, I think 10 on day 2 and then 8, 4. So yeah, there's some manpower behind the next-gen TG stuff. And that was our focus. I don't think anyone lifted a 2.0 finger, all the effort has gone into 2.1. With Luke Macken's help, we closed 21 tickets. I think we have all the road-blocks out of the way for the forthcoming b2 release, which will likely be the last beta before release candidates. The api is pretty solid now, I don't expect to expand it an more, so it's going to be feature frozen in b2. There are only 2 blocker tickets before 2.1rc1.
Mark Ramm and Jenny Steele overhauled the visual components of the 2.1 docs and came up with a better way of updating our public face. Jenny has also put forth an effort in c5t (a new cms based on mongo and tg2) to improve our image library capabilities. She also spent some time updating the TG admin so it looks way better now. I'll post some screenshots when I do the 2.1b2 release.
C5t got quite a bit of love actually. Rick Copeland added Ming 0.2 support since the api changed a bit from 0.1. Christopher Brown fixed a bunch of the styling. Kevin Mitchell added some tests. Jason Galyon worked on per-page authz, and Jorge Vargas worked on the editor implementation. I created a new c5t.website project so that we can work on the core and the public image independently. This is valuable especially since c5t does not ship with TW1, but we wanted to add mongodb support to the admin on the website, and TW1 is required for Sprox to do it's magic in the admin presently.
Kai Groner and Eric J (sorry I don't know your last name) both made some contributions to Sprox. Eric added Ming support after I refactored the database orm later organization in Sprox. Ming is a layer on top of pymongo that allows you to enforce schemata in your collections. This is perfect for selecting validators (the validator in the schema becomes Sprox's schema), and fairly simple to select widgets with. Eric had every provider test that works with SAORM working, except for one that looks like a pymongo bug on the limit operator. Kai worked on handling inline-forms in Sprox. Looks like most of the provider code is working, we just need to add in tw.dynforms in an intelligent way to hook the rest of the application up. While this is not a trivial task, it looks as if we are about half-way there for supporting inline forms. All of this work will go into sprox 0.7.
I also spent some time with Sprox, and added jquery support, but it's not ready for release. At the end of the sprints I had a working jquery table with pagination in the TG Admin, but some polish is going to be required to get that fully working. When I'm done, you will be able to easily swap between nothing, dojo, and jquery, and mootools support, or mix and match at any level. You can also switch between mongo, or any rdbms SA supports, and hopefully couchdb soon. It's kinda scary that's true.
01 Mar 2010 9:50pm GMT
28 Feb 2010
Planet Turbogears
Plumbling Life's Depths: PyOpenGL 3.0.1 Final (Test Please)
28 Feb 2010 9:40pm GMT
27 Feb 2010
Planet Turbogears
Plumbling Life's Depths: Much documentation revised
27 Feb 2010 7:08pm GMT
25 Feb 2010
Planet Turbogears
Christopher Perkins: Tutorial Day One, Sprint Day Two
One of the great things we have going with TurboGears is that the framework itself takes so little time to learn, that you can literally contribute on day two without much in the way of guidance. If you have web skills at all, you can be providing a positive input into the design and execution of TurboGears in a short time, say, 6 hours.
I'm returning from Pycon tomorrow, and I'll give a dump of that on Monday, but for now I need to write to promote my tutorial in Montreal. The conference has decided to provide the tutorial FREE (as in beer) to any Montréal-Python guy who goes to the conference, just use this link:http://confoo.ca/pymontreal.
Oh, and I am told that Yannick will be providing some homebrew for thoseof us who are sprinting, regardless of whether we attend the conference.
Day "2" sprint will be more like day 5, as the ConFoo.ca conference sits squarely in the middle, but in any event here is the announcement.
TurboGears has been confirmed for May 15 and 16 in Washington DC. Alex Clark has been helping me organize the event at Radio Free Asia. Please join us for a full-day TurboGears tutorial. We can explore topics like the TurboGears Admin, CRUD interface, even our Mongo Support. 2.1 will be out by then since we are nearing our second beta release.
Please spread the word about these important events, and I hope to see you all soon!
25 Feb 2010 6:36pm GMT
Plumbling Life's Depths: EJabberD Configuration some Other Day
Luke (of Moksha fame) has been guiding me through the "official" and correct way to do real-time web with TurboGears. Orbited seems to be a given, but STOMP apparently doesn't scale, while AMQP isn't well supported under Javascript. That seems to leave XMPP as the likely "official" way until AMQP is fully supported.
Of course, you need an XMPP server to do anything with XMPP, so I'm entering the maze of twisty-turny passages that is configuring EJabberD to connect to our TurboGears database for authentication. I figure I'll wind up with an external auth script for EJabberD which uses TurboGears command-line configuration to get the DB connection and use the standard friendly-form login table for authentication. Alternately I could go "wrong school" and just use an RPC call into a TG controller to do the auth, or maybe I could just use a "quick and light" XMPP server for the tutorial and let people struggle with EJabberD themselves (naw, that's not helpful).
However, today, before I get on the plane, I want to release PyOpenGL 3.0.1 final, so robust real-time web-apps for TurboGears are going to remain undocumented for now.
25 Feb 2010 4:58am GMT
24 Feb 2010
Planet Turbogears
Plumbling Life's Depths: That didn't go as well as I'd have liked
24 Feb 2010 3:11pm GMT
22 Feb 2010
Planet Turbogears
Plumbling Life's Depths: When the Floor is Crooked, You Don't Know How You Dance
Seem to finally have ChatTrack working on IE8. The key was installing Opera; it was only when that failed in the same way that I realize that it was the server-side code that was wrong (basically had < instead of >= (duh!)). Didn't cause a problem for FireFox or Chrome because they use the streaming code. I trust IE so little that I'd just written off the weird behaviour as an "IE Thing" and was trying to fix it that way (i.e. looking for weird JS failures and the like). I'd likely have spent hours more on that if I hadn't just taken a break to see what Opera would do. Likely won't get mobile versions done for PyCon, so no iPhones for now.
Now back to my presentation.
22 Feb 2010 5:21pm GMT
Plumbling Life's Depths: PyCon Dress Rehearsal Tonight (Tuesday)
22 Feb 2010 5:16pm GMT
Plumbling Life's Depths: PyCon Dress Rehearsal rocks the Casbah
Leigh and I did our presentations this evening to a pretty-much-packed house.
Leigh managed to keep pretty-much to time... I was way over (maybe double time), but then I knew I would be, the deck was almost 70 slides and that's before feedback, discussing ChatTrack as a project, etceteras, need to get the whole deck down to maybe 10-15 minutes so we have some time to get audience feedback. That shouldn't be a problem, this was, in essence, a first-draft deck, and it had a lot of fluff...
Need to remember to mention ChatTrack before I start talking, so that people actually get a chance to respond... just having the URL on every slide isn't enough, people think it's my company's URL. Even with that, fact is that my presentation, even the long version, is rather fast, it seems unlikely that we'll really get all that many people chatting/voting/feeding-back during it.
Myles gave me lots of good feedback on ChatTrack, and even tested on the iPhone for me (apparently it works there now, save for some CSS issues and voting, as that's a drag-and-drop thing and that doesn't seem to work on the iPhone). Nice guy, Myles.
No one seemed to have a better voting UI idea, but everyone seems to agree that the current UI is too non-standard for people to automatically understand. May wind up with icky up/down arrows (as I do for the whole-post up/down votes). All in all it seems like it's pretty much okay for now, with maybe a few tweaks before Saturday. One must-have is the links to the "highest-rated" and "most-commented" pages, as without that you can't pull up the "summary" during the presentation to respond.
Tomorrow I'm hoping to spend with Soni and Lex for the most part (as well as getting packed and all the rest). I'm on an early-afternoon flight to Atlanta, so hopefully I'll get to hang out with some people on Thursday night.
22 Feb 2010 5:36am GMT
19 Feb 2010
Planet Turbogears
Plumbling Life's Depths: 3D/Graphics Open Track at PyCon
Okay, we now have our own Open Space Wiki Page for figuring out what 3D/Graphics related stuff you want to do at PyCon. If you're interested in 3D or graphics/games stuff, edit the page and brainstorm about what you'd like to see or work on. I'm happy to do impromptu classes on whatever OpenGL-ish topics people would like to learn about (if any). Co-coding on games, libraries, engines, etceteras would also be fun. Having people with engines guide people through using them (or even how they work) would be fun too.
19 Feb 2010 9:30am GMT
17 Feb 2010
Planet Turbogears
Plumbling Life's Depths: The "Shiver" Moment in Presentation Writing
There's a moment in writing presentations; you are dispassionately writing and editing point-form notes about the things you want to talk about, a kind of disjoint series of ideas that you know all fit together somehow, and you're really just playing with them to see how they fit, then you take a bit of time off to help someone on IRC and you come back to it.
There, staring you in the face is "the" presentation, hidden among your ridiculously over-detailed notes, trying to push through the mess with a simple, clear, straightforward narrative, something that needs to be said. You can suddenly see the underlying ideas that made the whole set of comments feel necessary, and now you know you need to toss out everything you have... but the message has finally shown up, and thus all is right in the world.
I love that moment.
17 Feb 2010 8:00am GMT
15 Feb 2010
Planet Turbogears
Gustavo Narea: WSGI and Repoze on identi.ca
A few weeks ago I started the Repoze and WSGI groups on identi.ca. Please feel free to join us if you have something to share! Or just keep an eye on those groups to receive updates.
15 Feb 2010 9:56pm GMT