Thanks for attending my talk if you were able to make it over to Abington. I had to leave after my presentation so I didn’t get a chance to see any of the other presentations, but the consensus appears to be that it was another great event. The slides are posted below for anyone that wants to reference them.

I was asked to take a few photos for this event by fellow pythonista Tom Panzarella. I think they turned out OK for the most part – although the EF 70-200 L IS II saved my arse in the end. It was difficult shooting with the lighting setup at the venue, but I’ll be back there to get some more practice.

All of the photos are over on Flickr and they were even linked to by Technically Philly!

This is mainly for me own reference, but it is probably something that other people will find useful. If you attempt to use the rlcompleter module on the Mac you’ll find that tab completion does not work correctly. Apparently this is because Apple ship a neutered version of readline.

To get completion working you have to use a different argument to readline.parse_and_bind(). The base example uses:

readline.parse_and_bind("tab: complete")

Change that line to:

readline.parse_and_bind ("bind ^I rl_complete")

I’ve spent a fairly big chunk of my professional career building public websites for a number of clients. One of the things that comes up again and again is search engine optimisation.

Surprisingly there is very little coverage of this is in business and marketing journals. Patrick Reid wanted to start raise awareness, and needed my technical help in preparing a paper. We finished it up earlier this year and it was published in the Journal of Strategic Management Education.

Corporate Communications in the FTSE 100: Evidence using Search Engines
JSME Vol 7: Issue 1, 2011
Patrick Reid (Greenwich School of Management, UK), Brian Lyttle (LiquidHub, Inc., USA)

Search engines have become an essential tool for information seekers on the world wide web. This makes it critical that company websites are optimised to achieve the highest possible ranking across the major search engines, in particular Google, ensuring maximum exposure and subsequent awareness. This paper identifies key factors in search engine optimisation and then analyses a sample of the largest 10 firms listed on the London Stock Exchange.

The findings reveal a lack of adoption of best practices which will impact on their search rankings. The paper also discusses tradeoffs inherent in search engine optimisation and draws out key implications for managers and academics.

The full paper is available from the Senate Hall site.

Following the recent PhillyPUG meetup I was trying to install scrapy on an old MacBook Pro running OS X 10.6 (Snow Leopard) and ran into a number of problems with the lxml dependency. This is the parser used to extract data from pages that scrapy downloads so you are not going to get very far without it.

It seems that the compilation problems experienced when installing with pip result from an attempt to build a universal binary. If you have Xcode 4 installed then you lose some of this capability and need to make sure that the correct architecture is specified.

Architecture Fix

Setting the architecture is something you can do in your bash profile, executing it under a new bash ensures that the build script picks it up.

sudo bash
export ARCHFLAGS='-arch i386 -arch x86_64'
pip install lxml # test it
pip install scrapy --upgrade # fix the failed scrapy install

Original Error

brianly$ sudo pip install lxml --upgrade
Downloading/unpacking lxml
  Downloading lxml-2.3.tar.gz (3.2Mb): 3.2Mb downloaded
  Running setup.py egg_info for package lxml
    Building lxml version 2.3.
    Building without Cython.
    Using build configuration of libxslt 1.1.24
    warning: no previously-included files found matching '*.py'
Installing collected packages: lxml
  Found existing installation: lxml 2.2.2
    Uninstalling lxml:
      Successfully uninstalled lxml
  Running setup.py install for lxml
    Building lxml version 2.3.
    Building without Cython.
    Using build configuration of libxslt 1.1.24
    building 'lxml.etree' extension
    gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch ppc -arch x86_64 -pipe -I/usr/include/libxml2 -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c src/lxml/lxml.etree.c -o build/temp.macosx-10.6-universal-2.6/src/lxml/lxml.etree.o -w -flat_namespace
    /usr/libexec/gcc/powerpc-apple-darwin10/4.2.1/as: assembler (/usr/bin/../libexec/gcc/darwin/ppc/as or /usr/bin/../local/libexec/gcc/darwin/ppc/as) for architecture ppc not installed
    Installed assemblers are:
    /usr/bin/../libexec/gcc/darwin/x86_64/as for architecture x86_64
    /usr/bin/../libexec/gcc/darwin/i386/as for architecture i386
    src/lxml/lxml.etree.c:161594: fatal error: error writing to -: Broken pipe
    compilation terminated.
    lipo: can't open input file: /var/tmp//ccYr9GpX.out (No such file or directory)
    error: command 'gcc-4.2' failed with exit status 1
    Complete output from command /usr/bin/python -c "import setuptools;__file__='/Users/brianly/dev/github/pyconscrape/build/lxml/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-axeEA7-record/install-record.txt:
    Building lxml version 2.3.

Building without Cython.

Using build configuration of libxslt 1.1.24

running install

running build

running build_py

creating build

creating build/lib.macosx-10.6-universal-2.6

creating build/lib.macosx-10.6-universal-2.6/lxml

copying src/lxml/__init__.py -> build/lib.macosx-10.6-universal-2.6/lxml

copying src/lxml/_elementpath.py -> build/lib.macosx-10.6-universal-2.6/lxml

copying src/lxml/builder.py -> build/lib.macosx-10.6-universal-2.6/lxml

copying src/lxml/cssselect.py -> build/lib.macosx-10.6-universal-2.6/lxml

copying src/lxml/doctestcompare.py -> build/lib.macosx-10.6-universal-2.6/lxml

copying src/lxml/ElementInclude.py -> build/lib.macosx-10.6-universal-2.6/lxml

copying src/lxml/pyclasslookup.py -> build/lib.macosx-10.6-universal-2.6/lxml

copying src/lxml/sax.py -> build/lib.macosx-10.6-universal-2.6/lxml

copying src/lxml/usedoctest.py -> build/lib.macosx-10.6-universal-2.6/lxml

creating build/lib.macosx-10.6-universal-2.6/lxml/html

copying src/lxml/html/__init__.py -> build/lib.macosx-10.6-universal-2.6/lxml/html

copying src/lxml/html/_dictmixin.py -> build/lib.macosx-10.6-universal-2.6/lxml/html

copying src/lxml/html/_diffcommand.py -> build/lib.macosx-10.6-universal-2.6/lxml/html

copying src/lxml/html/_html5builder.py -> build/lib.macosx-10.6-universal-2.6/lxml/html

copying src/lxml/html/_setmixin.py -> build/lib.macosx-10.6-universal-2.6/lxml/html

copying src/lxml/html/builder.py -> build/lib.macosx-10.6-universal-2.6/lxml/html

copying src/lxml/html/clean.py -> build/lib.macosx-10.6-universal-2.6/lxml/html

copying src/lxml/html/defs.py -> build/lib.macosx-10.6-universal-2.6/lxml/html

copying src/lxml/html/diff.py -> build/lib.macosx-10.6-universal-2.6/lxml/html

copying src/lxml/html/ElementSoup.py -> build/lib.macosx-10.6-universal-2.6/lxml/html

copying src/lxml/html/formfill.py -> build/lib.macosx-10.6-universal-2.6/lxml/html

copying src/lxml/html/html5parser.py -> build/lib.macosx-10.6-universal-2.6/lxml/html

copying src/lxml/html/soupparser.py -> build/lib.macosx-10.6-universal-2.6/lxml/html

copying src/lxml/html/usedoctest.py -> build/lib.macosx-10.6-universal-2.6/lxml/html

creating build/lib.macosx-10.6-universal-2.6/lxml/isoschematron

copying src/lxml/isoschematron/__init__.py -> build/lib.macosx-10.6-universal-2.6/lxml/isoschematron

creating build/lib.macosx-10.6-universal-2.6/lxml/isoschematron/resources

creating build/lib.macosx-10.6-universal-2.6/lxml/isoschematron/resources/rng

copying src/lxml/isoschematron/resources/rng/iso-schematron.rng -> build/lib.macosx-10.6-universal-2.6/lxml/isoschematron/resources/rng

creating build/lib.macosx-10.6-universal-2.6/lxml/isoschematron/resources/xsl

copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl -> build/lib.macosx-10.6-universal-2.6/lxml/isoschematron/resources/xsl

copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl -> build/lib.macosx-10.6-universal-2.6/lxml/isoschematron/resources/xsl

creating build/lib.macosx-10.6-universal-2.6/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl -> build/lib.macosx-10.6-universal-2.6/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl -> build/lib.macosx-10.6-universal-2.6/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl -> build/lib.macosx-10.6-universal-2.6/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl -> build/lib.macosx-10.6-universal-2.6/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl -> build/lib.macosx-10.6-universal-2.6/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt -> build/lib.macosx-10.6-universal-2.6/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

running build_ext

building 'lxml.etree' extension

creating build/temp.macosx-10.6-universal-2.6

creating build/temp.macosx-10.6-universal-2.6/src

creating build/temp.macosx-10.6-universal-2.6/src/lxml

gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch ppc -arch x86_64 -pipe -I/usr/include/libxml2 -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c src/lxml/lxml.etree.c -o build/temp.macosx-10.6-universal-2.6/src/lxml/lxml.etree.o -w -flat_namespace

/usr/libexec/gcc/powerpc-apple-darwin10/4.2.1/as: assembler (/usr/bin/../libexec/gcc/darwin/ppc/as or /usr/bin/../local/libexec/gcc/darwin/ppc/as) for architecture ppc not installed

Installed assemblers are:

/usr/bin/../libexec/gcc/darwin/x86_64/as for architecture x86_64

/usr/bin/../libexec/gcc/darwin/i386/as for architecture i386

src/lxml/lxml.etree.c:161594: fatal error: error writing to -: Broken pipe

compilation terminated.

lipo: can't open input file: /var/tmp//ccYr9GpX.out (No such file or directory)

error: command 'gcc-4.2' failed with exit status 1

----------------------------------------
  Rolling back uninstall of lxml
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.6/site-packages/pip-1.0.1-py2.6.egg/pip/basecommand.py", line 126, in main
    self.run(options, args)
  File "/Library/Python/2.6/site-packages/pip-1.0.1-py2.6.egg/pip/commands/install.py", line 228, in run
    requirement_set.install(install_options, global_options)
  File "/Library/Python/2.6/site-packages/pip-1.0.1-py2.6.egg/pip/req.py", line 1104, in install
    requirement.rollback_uninstall()
  File "/Library/Python/2.6/site-packages/pip-1.0.1-py2.6.egg/pip/req.py", line 487, in rollback_uninstall
    self.uninstalled.rollback()
  File "/Library/Python/2.6/site-packages/pip-1.0.1-py2.6.egg/pip/req.py", line 1417, in rollback
    pth.rollback()
AttributeError: 'str' object has no attribute 'rollback'

Storing complete log in /Users/brianly/.pip/pip.log

I wasn’t a big fan of so-called Fluent Interfaces but my fondness is growing as I use Fluent NHibernate on an ASP.NET MVC project. When I started on my project I checked out the repository pattern example from Google Code to see how this was being implemented. The supplied repository interface provided the following interface:

public interface IRepository
{
    T Get(object id);
    void Save(T value);
    void Update(T value);
    void Delete(T value);
    IList GetAll();
}

Looks great apart from the fact that you need to pass an entity to the Delete() method, which would in turn result in iteration over a large number of objects if I had to do a lot of deletes. If I received an integer with an ID from an ASP.NET MVC controller I’d have to retrieve the object before sending it to the Delete() method. This is not very efficient.

To remedy this problem I added an additional Delete() method to the interface taking an object ID as the only parameter:

public void Delete(object id)
{
    using (var session = sessionFactory.OpenSession())
    using (var transaction = session.BeginTransaction())
    {
        var queryString = string.Format("delete {0} where id = :id", typeof(TEntity));
        session.CreateQuery(queryString)
               .SetParameter("id", id)
               .ExecuteUpdate();

        transaction.Commit();
    }
}

This is a very simple method that will generate more efficient SQL under the covers for deletion of objects from the database.

ASP.NET MVC provides a way for developers to extend and create custom ActionResults for controllers. Normally you return a view from a controller and the runtime goes off and hunts down the appropriate view based on the name of your action. This make sense for normal web pages but sometimes you need to build something in code or you don’t want to write a view for it. In this case you can extend ActionResult, set a content type, and render the result to the output stream.

public OpenLayersActionResult MapData()
{
    var myPoints = new List();

    var p = new MapPoint
                {
                    Title = "Title for point on the map",
                    Icon = "marker.png",
                    IconSize = "21,25",
                    LatLon = "12.9715987,77.5945627",
                    Description = "This is the long description for a point on the map"
    };

    myPoints.Add(p);

    return new OpenLayersActionResult { Points = myPoints };
}

Over on GitHub you can find some sample ActionResult classes including one that replicates the built-in JsonResult. The one that will probably be most useful to someone is the OpenLayersActionResult which simplifies generation of layer data. I’m digging through various projects and building out that library with other helper code from various projects so keep an eye out for more snippets.

Thanks to everyone who attended my talk. I hope it was useful! If you haven’t done so already please complete an evaluation form.

I’ve uploaded the presentation to SlideShare and you can download a PDF from my site too. The code sample I used during the presentation is available on BitBucket, and I’m going to be making various improvements to it over the coming weeks.

If you want need a 64-bit version of this module for Google App Engine, or another project, you can run into a couple of issues. Sticking to 32-bit versions of Python and PIL avoids these issues so that may be the best direction for newbies.

The downloads on the PIL site are for 32-bit versions of Python which means that you hit runtime issues as soon as the Python script files attempt to load the 32-bit libraries into 64-bit Python. Annoyingly the installer is unaware of the architecture for the Python installation it finds and will leave you with a broken PIL install. Just uninstall it from Control Panel. My first thought was to try an build from the sources but I figured someone has run into this problem.

After some searching I found a site which offers pre-built versions of many Python modules for 64-bit architectures including PIL. If you are in any way concerned about performance, or security a better solution might be to build from source or use versions from a company offering supported versions.

I downloaded the version for Python 2.5 and then discovered that it would not install. What is interesting is that the 32-bit installer was able to find my 64-bit Python, but the 64-bit installer was unable to find it.

After some more searching it turns out that the installer takes it’s cue from a key in the registry and the Registry virtualization in Windows x64 was confusing it. To resolve the issue I had to export the “HKLM\SOFTWARE\Wow6432Node\Python\PythonCore\2.5” key, remove the “Wow6432Node\” string using a text editor, and re-import the key so that it was at “HKLM\SOFTWARE\Python\PythonCore\2.5”.

After doing this the 64-bit installer for PIL detected my Python installation and I was up and running.

Van Lindberg recently tweeted a link to some tips on how to attend a conference. The advice in that article is pretty sound for a business person looking to build up their network to find sales opportunities, but it doesn’t cover any PyCon-specifics.

After a little thinking I’ve come up with my own list of tips for new PyCon attendees:

Attend some open space sessions

Open spaces are now happening at a lot of conferences but the PyCon ones have been the best I’ve seen. These are ad hoc discussions on a common topic with participants joining and leaving whenever they want. These sessions are a lot of fun and you’ll make a lot of friends in the process. Even if you aren’t the most talkative person, or know little about a subject, you are free to just hang out and listen.

Whilst there is a page on the PyCon website for these sessions, it is normally best to check the boards outside the rooms assigned to open space use. You should also monitor the Convore backchannel and #pycon on Twitter.

If you don’t see any open space sessions related to your favourite topic just post a session in one of the available time slots. In almost all cases some likeminded individuals will turn up.

Scheduled talks are normally recorded for viewing later so I’d recommend that you try to decide ahead of time which talks are must see, and which ones could be skipped. That way you can make the most of the open space sessions as some of these will overlap with scheduled talks, especially on Saturday and Sunday.

Help the PyCon staff

Putting on a conference PyCon is an enormous effort. They are often looking for runners and other helpers. Spending a little time helping out introduces you to many interesting people and you’ll probably learn a bit too about how conferences run.

Follow @pyconvolunteers, talk to the organisers, and lurk on the pycon-organizer mailing list.

Check out the lightning talks

These are very short talks on a variety of Python-related subjects scheduled at the beginning and end of conference days. If you have something to say to the community they are a good way to let people know about your project or passion. For everyone else they are a good way to find out about lesser known projects, and they are often quite humorous.

Go off track

PyCon like many other conferences has tracks with related content and there are 10 virtual tracks for 2011. These are really helpful for finding things that you are likely to be interested in, but sometimes it is good to check out something a little bit different.

In particular, veteran PyCon speakers like Alex Martelli, David Beazley, and Raymond Hettinger fill rooms each year and are always worth watching.

Chat with presenters

On the topic of presenters there are a lot of opportunities to talk to presenters at PyCon, either right after a talk, or later at the conference. If you have questions on their area of expertise or feedback on their talk it is worth taking some time to talk to them.

Spend time on things that are valuable to you

Sometimes you’ll end up in a talk or open space session that isn’t exactly what you expected. It is best if you go to something more to your taste than waste your time on a talk that is too basic or not as described. Sometimes this can be a little difficult if a session is packed with people, but you can do things to make your exit a little bit more comfortable.

First of all if you have an inkling that you are going to want to leave a talk don’t pick a seat in the middle of a row. Instead sit somewhere that that has easier access to the exit at the back of the room. If there are video cameras in the talk you might not want to be "the dude that left in the middle of a talk". In reality you will likely not be noticed as the cameras are directed at the speaker.

Second, make sure to pack up your laptop and other things in your bag before exiting. It’s kind of annoying to get clobbered with a laptop as someone exits past you. The same thing goes for anyone sitting in the row. Don’t make unnecessary obstacles for anyone that gets out. Move your power cord and move closer to power strips rather than pulling them out of place.

Find a group for dinner in Atlanta

Watch the Convore and Twitter for information on groups who are going out for the evening in Atlanta. In many cases an open space session will continue over dinner!

Don’t have too good a time at night

It is way too easy to have a good time on Friday and Saturday nights, but don’t stay out too late or get too hammered. This seems like a bit silly but you need quite a bit of energy to make it all the way through a conference like PyCon.

I’m sure there are many things that I’ve missed here. Please post other tips on Twitter under the #pycon hash tag, on your blog, or post a comment below for others to read.