jp's domain

Over the past year or so I have spent a whole lot of time evaluating different web frameworks, trying to find what works best for me. Like many programmers, I am a bit of a perfectionist, but I am also a pragmatist. I want my tools to work absolutely right every time, and I want my tools to help me effectively get certain tasks done. However, if you've programmed much at all, you know that perfection isn't something that you are likely to find, as perfection to one person is different to what perfection for another person is. In the end you can only pick what works best for you.

Compounding this decision process is the fact that in Python web development, there are quite a few choices. Now some people have criticized Python for this abundance of options, but I do not believe it is in a bad thing. We have seen how in the Ruby community, since there is realistically only Rails, Rails tends to dictate what is 'good' and what isn't. Rails itself hasn't added a lot of new functionality over the past few years. On the other hand, the numerous frameworks in Python have provided different ways of looking at things and it has ended up creating numerous cycles of improvement on existing components. This ultimately can only be good for the Python web community. If it ever boils down to there being a single framework, things will probably grow stagnant, but the forces of multiple frameworks will only improve all that are involved. But this doesn't help the programmer that is right now, in 2007, looking for a good tool to use. In this post I hope to make an objective analysis of the different frameworks available in the Python community and to hopefully give the reader a better idea of what tool will work for them.

I'm going to break this article down into 5 main sections: Zope and web.py, roll your own framework, Django, Pylons, and TurboGears. In each individual section (I don't do this with the first), I will break down each framework by:

• the general philosophy of the framework
• the URL dispatching system used
• the templating system used
• the object-relational mapper used
• the form handling used
• extra types of functionality
• advantages of using the framework
• disadvantages of using the framework
• the future direction of the framework and concerns I have with it

I attempt to be as fair and objective as possible, as well as to be critical while remaining positive.

Note: The first time I mention a project I provide a link, but because I refer to different projects multiple times, the first time is the only time I will link it. You can scroll back and find the link or just use Google. Sorry, I am lazy. :)

Secondary note: This is by no means a scientific analysis and it includes my own opinions and and mistakes. Hopefully people will speak out and correct them and bring forward their own opinions in the Reddit comments tied to this link.

First, something big and something small

For most users, I think there are 3 main frameworks worth checking out, that is: Django, Pylons, and TurboGears. But there are also a few alternatives, and I would like to briefly mention them. First of all there is Zope. It is what Python web development at one time was instantly associated with. Now you don't hear it discussed nearly as much, and for many people it might seem to be irrelevant. But Zope still has a good-sized community, and it also has great support, so it is perhaps the 'safest' choice. Also, the proliferation of frameworks recently have caused Zope to focus on modularization and simplification, and a lot of Zope components are being split out into eggs of their own. It is possible that Zope components might at some point become something that people use regularly within the other frameworks. Somebody who just wants to build a simple blog or other basic application would probably find Zope overkill, but a bigger company may find it very useful.

In addition to Zope there is web.py. Now web.py is on the extreme other end of the spectrum from Zope, and was originally just a single file released by Aaron Swartz at the end of 2005 after being written to power Reddit.com. Its foremost goal is simplicity and many people like it for that. It appears that lately certain features like a templating language has been added, and it is a great choice if you just want something to hack around with or you just want something that is absolutely miminal. Unfortunately it probably has the smallest amount of users of any of the more recent frameworks, and most people need their framework to do a little more. It also probably wouldn't be the best choice to build company applications with.

Roll your own

However, just because you need a little bit more functionality out of your framework, it doesn't mean you have to use one of the larger ones. Python has a plethora of different components that anyone can plug together to create their own little custom framework to fit their exact needs (infact these components are used throughout the larger frameworks and there is nothing stopping you from pluggin them into Zope or web.py).

Philosophy

Who knows what is best for me or my app besides me?

URL dispatch

Routes (Rails-style URLs), selector (feels a lot like Django's URL system, but a bit simpler), and more fully-featured systems like colubrid (multiple dispatch styles), CherryPy (object-publishing and more), or Spyce

Templates

Genshi (XML-based), Jinja (Django-style templates), the venerable Cheetah, Mako (about as close as you can get to Python in templates; the successor to Myghty), and others

ORM

SQLObject, the popular SQLAlchemy, interesting sytems like Schevo, and useful tools like Migrate and Elixir (an SQLObject-ish declarative layer to be used on top of SQLAlchemy)

Form handling

FormEncode (form validation and data conversion), FormBuild, Toscawidgets, and WebHelpers (Rails-style helper functions)

Extras

And finally for the remaining components like WSGI request/response objects, WSGI middleware, app creation and deployment, there is Paste and its libraries including Paste Script and Paste Deploy as well as smaller projects like Yaro. In addition to all these components you can theoretically use the different parts of Django in your own framework.

Advantages

Basically there is nothing stopping you from just putting something together yourself, and this may satisfy some of the more hacker-minded types out there. Joe Gregorio has written an excellent article describing how he put together his own 'throw away' framework in about 3 hours and about 60 lines of Python code. To me that is a very powerful testament of just how good the different framework components are. And what is even better is most of them have excellent documentation.

Disadvantages

As Joe warns, this approach doesn't buy you everything, "We don't have an instant admin interface, we don't have generic views, automatic form generation, automatic form handling, the Django community, bug tracking, IRC, etc, etc." Community, support, and already-tested code shouldn't be overlooked. And then there are the little things. Is your framework thread-safe? If your application gets used a lot, does your framework integrate nicely with memached like the major frameworks? Rick Copeland has written an interesting piece detailing the problems in writing your own framework.

Future and concerns

Rolling your own framework will never go away. It is a very attractive option and can work well in certain situations. I think if Paste Deploy and its sister libraries pick up more steam, and as modularization of components continues, this will be seen more and more. Hopefully with WSGI as a basis, this will act as a boon to innovation and not a bane. Ideally it shouldn't matter exactly how you create your app, as long as it integrates nicely along with other WSGI applications there isn't an issue. We see this already with some people running the Django admin tool on top of applications written in other frameworks.

Django

Note: 0.96 released a few hours after I got done writing this. It shouldn't affect much here other than when referring to 0.96 in the future tense.

So in alphabetical order, we shall start with Django. Now in the past I've been overly critical of Django and I don't really think that is fair, because my criticism was more over different ideologies in design rather than the quality of the framework itself. Django is a solid project that will provide really everything you need in a web framework except for AJAX (unlike Rails in this respect; Django leaves this up to you). Django is without a doubt one of the more vibrant projects in the Python community.

Philosophy

You can't make everyone happy, but you can focus on providing a complete set of components that is well-documented and integrated.

URL dispatch

Django's dispatching is based around pure regular expressions. You write a set of regular expressions and when your application is triggered, it searches from top to bottom through the regular expressions to see if the URL matches one of them. It then dispatches to a function that you provide. This approach is both extremely flexible and fairly simple, at least conceptually. Most URLs aren't too complex, yet if you have a legacy application it is very easy to integrate Django with it. In addition the mapping of URLs to pure functions allows some neat things like the wrapping of these functions in common bits of funtionality (such as creating an object, listing an object, deleting an object, etc), in what Django calls generic views.

Templates

The entire idea behind Django's templates is to keep them as simple as possible. As has been stated several times, the last thing the developers want to do is just to create another programming language. Django templates provide minimal functionality because they are not designed for application logic, they are designed for formatting. As a result Django's templates work very well in a situation where programmers and designers work separately. Programmers who do both programming and design might feel a little handicapped by these, however, which isn't necessarily a bad thing as they do limit the application logic you can do in the template itself (it shouldn't be there). New functionality is added via tags and filters; tags providing more complex functionality such as looping, filters acting much like UNIX pipes which change the value or appearance of a variable in some way (and yes, like UNIX pipes you can chain several together). Django provies quite a few tags and filters out of the box, and creating simple ones isn't difficult. Django also has a very good inheritance system which makes it easy to create a base template that contains specific parts like header, body, sidebar, and footer that children templates can fill in. If you do find yourself wishing for a bit more power in your templates, theoretically it shouldn't be that difficult to hook up another templating system to Django.

ORM

Django's ORM allows you to define your database objects in pure Python (this will be familiar to those who have used SQLObject before). This is very simple and works pretty well in most cases. I haven't really found anything that the Pythonic-query language cannot handle, and it is enjoyable to work with. Django's ORM is not the most powerful thing out there, but it does provide functionality such as database introspection (if you already have a legacy app with a database), and transactions. There has been talk for more than a year now in building this Django DB API on top of the very powerful SQLAlchemy, but it could be some time before that is completed. In the meantime the current ORM works well in most cases, and provides support for several DB systems.

Form handling

The form handling system in the current release (0.95.1) is decent if a bit cumbersome. Thankfully this will be changing in 0.96 with the newforms system. newforms makes defining your forms feel a lot like defining your database objects (basically easy and Pythonic). Newforms will let you design your form in this manner, and then it will display it, deal with validation errors (you specify what type of validation is required), and either redisplay the form as needed or process the data by creating a new object, editing an existing one, whatever. Newforms also does a good job of separating display logic (widgets) from the form processing itself, and one of the early examples provides a TinyMCE rich Javascript control as a widet used in the place of a regular textbox. newforms makes dealing with forms so much easier, and quite simply frameworks such as Rails have nothing like it. This is one of the most awaited features of Django and for good reason. A super simple form is given below (I am being a little lazy and not looking up anything to verify that it is right, so this isn't 100% accurate):

class TestForm(Form):
    name = CharField(maxlength=20)
    phone = PhoneNumberField(required=False)
    email = EmailField(initial='...@gmail.com')

----

def view(request):
    if request.method == 'GET':
        form = TestForm()
    else:
        if form.is_valid():
            ...do magic processing here...
    return render_to_response('/template.html', {'form':form})

----

<h1>This is my form</h1>
(( form )) (should be double braces, but the formatting is messing that up)

Extras

Individual Django programs are termed 'projects' while pieces of those programs are 'apps'. Apps provide small bits of functionality, for example you might have an app for comments, an app for user registration and authentication, or an app for polls. This creates some pretty neat plug-and-play possibilities, and in fact Django comes bundled with several apps that you are free to use in your own projects. These applications by themselves give Django more functionality out of the box than most frameworks, and are why Django in some ways is seen as similar to a content management system. Perhaps Django's most discussed app is the admin app which provides a basic backend for authenticated users to enter data. It has a decent amount of flexibility, and that flexibility will be be getting even better when the admin app gets based around newforms.

Advantages

Django has by far the largest community out of any of the the major Python frameworks. The IRC channel and mailing lists are fairly friendly and it is easy to get a response to a question you might have. Django also has perhaps the best documentation out of any web framework in existence. This may be a bit of an exaggeration (though I haven't come across a framework with better docs), but it is a definite focus of the developers which is comforting for many users, especially those just getting started with a framework. In addition a book is being written which is and will be available online for free and will be put into print by Apress. Due to this documentation and the aforementioned included applications, you can get quite a bit done with Django in a short amount of time. In addition, one of my favorite things about Django is everything is kept very simple and Pythonic. The configuration files are pure Python, URLs map to simple functions, and you define your database objects in Python as well. If you know Python you should be able to pick up Django pretty quickly.

Disadvantages

The only real disadvantages one might come upon when using Django is if you don't agree with it. Django isn't designed for customization to be particularly easy, so if you don't agree with the URL dispatching, the templating language, the ORM, or anything else, while it is possible to switch them out, Django isn't going to hold your hand. This is kind of compounded by Django's components not being super powerful, so for example you might run into problems using Django's ORM with a large and complex legacy DB. Issues like this aren't things most users will run into, though.

I did find that Django got repetitive on a few counts. First of all I'm not good with regular expressions, so defining URLs led to a lot of cutting and pasting for me. This approach is also pretty error prone (oops, I forgot the ^ at the beginning of my URL!) and it isn't especially easy for new users (which flies in contrast to how Django is in almost every other component). It would be nice if there was a slightly higher-level wrapper available for URLs. I'm also not the biggest fan of the templating language. While it does its job well (that is to not be another half-done programming language), filters and tags aren't nearly as easy to write as the simple functions are that you might write to use in something like Genshi or Mako. But this is a minor problem that you can for the most part look past.

I also ran into little bugs here and there. One bug made uploading large files crash duplicate the file in memory and therefore crash the server, one bug meant that the time zone setting in my settings.py file was off by 3 hours, and a further one was a bizarre bug way down in the bottom of the ORM which led to some unexpected results. Again, these are little things, and they should be fixed over time.

My final issue was the 80% rule. It seemed to me that in many cases Django would almost do what I need but not quite and therefore I'd have to give up using a higher-level component and instead do whatever it was manually. I ran into this with both the admin application and generic views, which are ideally nice features that I would really like to be able to use. Perhaps the best way of looking at both of them is to not expect the moon from them; both have particular goals and purposes, and they aren't made to do everything. Regardless, I would hope that in the future more emphasis is placed on making these higher-level features as flexible as possible (and gladly I do see that in something like the newforms-admin branch).

Future and concerns

The immediate future looks toward the release of newforms in 0.96 as well as later additions like the newforms-admin. The SQLAlchemy branch which layers the Django ORM on top of SQLAlchemy is also worth looking forward to. Another database-centric addition is a migrations tool, which is something a lot of people have requested and something that has been worked on (known as SchemaEvolution in the Django community). For the long-term you can expect the release of Django 1.0 and it is then that the developers have discussed finally 'marketing' Django; considering that they haven't officially done so already, it is remarkable just how popular Django has gotten (really it speaks to the quality of what they've put together). There is also talk of smaller changes like modifying Django's internals to use Unicode throughout and of automatically escaping variables in templates.

My concerns with Django are mainly theoretical in nature. With Django being the most popular project in Python web programming by far, there is a bit of a responsibility to do things which will further the web community as a whole. As I mentioned earlier, it is precisely this competition between frameworks which has led to such great innovation; if Django takes steps which locks users into using the framework and keeps other frameworks from having access to certain tools, the Python community could become too focused on a single framework and things could stagnate (which is what I personally feel has happened in the Ruby world).

Before I start on the theoretical issues, I'd like to deal with practical concerns in the Django community. First of all, the entire project/application model is one of Django's most powerful features, this really needs to be harnessed in some way. Currently it is pretty difficult to find applications created by other users, and there ends up being a lot of duplicated effort. If there was an official repository of Django apps, tags and filters for templating, and middleware, a lot more effort could be focused on fixing what is existing instead of invididuals having to create their own libraries which do the same thing. Any such repository in my opinion should be based around Python eggs for easy installation from the Python Cheese Shop (there is no reason to reinvent the wheel here). This to me is perhaps the most important thing that can happen with Django. There is a massive community out there with a lot of potential code to contribute that is more or less being wasted right now.

In addition, the very structure of the Django project perhaps needs to be analyzed. There are major bugs or features that have lain in an unfinished state for over a year now. The SQLAlchemy branch was announced in August of 2006, there still has been almost no progress on it. SchemaEvolution was a 2006 Summer of Code project, it, too lies in an unfinished state and there seems to be no more progress in making it an official part of Django than there was 6 montshs ago. The bug I mentioned earlier which saw files duplicated in memory (a fairly major issue for anyone doing file uploads in a web application) was something I ran across in August of last year, but it had actually had a bug report (this ticket leads into 2070) opened over a year ago. A fix still hasn't been made available in any Django release. Things like this to put it bluntly are unacceptable because they make Django unusable for certain people. Numerous other Python frameworks have gotten file uploads right, I don't understand why it has been so difficult for Django to do so. This leads into another practical issue I have with Django: the project needs to look at using outside libraries wherever possible. If Paste does file uploads very well already, why not use its implementation? This means the actual Django developers have to worry less about small components like this and more on the overall design of the framework. In addition it means other people are outside of Django are working on improving that component. Less work, less bugs, and for Django essentially a smaller codebase. Now certain outside libraries simply will not fit Django's design, but when they do they really should be utilized, especially with something as fundamental and important as file uploads.

Things just seem to move way too slowly in the Django community. It may be simply due to the size of the project, but there is a slight feeling that the community itself does not have a lot of involvement in the further development of the framework. There's a bit of a disconnect between the developers and the community. I know that I can drop into the Pylons chat room and talk to Ben Bangert the creator of the framework, or talk to Mike Bayer the creator of SQLAlchemy and Mako and say, "Can we look at improving this?", "Can we fix this bug?", "Can you describe how I might plug in this component to the framework?". A lot of times even if they don't agree with my suggestion (this happens a lot; haha), they will explain to me what is going on, or if there is a bug I've literally seen them fix it in the codebase on the spot. I've never had this experience with Django. I've never even seen the lead developers inside the chat room. Now I can understand if they do not have to the time to do so, but I would really like it if there were a community-dev who could be around to deal with issues like this and to for example, hear and get moving on a solution to fix something like the file uploads. Again, Django has a large and enthusiastic community. Harness it! Note: This got posted to Reddit a bit sooner than I was ready. I talked to James Bennett earlier tonight, he is the release manager and also has commit rights to the source code. In many ways he can be seen as the 'community-dev' that I am referring to. He was telling me that recently the community involvement has gotten better. And what do you know, not but 5 minutes after I got talking to him another dev dropped into IRC. Hopefully this developer/community reaction does continue to improve, there really needs to be quicker turnaround on fixing errors. Also it should be noted that Django started out as a proprietary project that became open-source, so it is understandable that this is an ongoing process.

Now back to the theoretical: I would really like to see Django focus on modularization. The easier it is to use Django's components with other frameworks (such as the URL dispatcher, the templating system, the ORM), the more people will end up using them, and more bugs will be spotted and more functionality added (like more tags and filters for the templating system). If this were the case, time spent duplicating functionality to be easily used outside of Django, such as Jinja could instead be used to work on what already exists. This is a win for everyone involved. In addition I'd like to see it made easier to use components from outside Django inside Django itself. For example TurboGears and Pylons both allow you to change a few lines of code to use a different templating system. Integrating that and functionality like this into Django would be very powerful and would alleviate the issues some people (myself included) have with specific Django components.

The support of specific standards and currently existing libraries in the Python web community would also be a net gain for everyone involved. As it stands currently, while an app such as the user application in Django might be very useful to a Pylons user, apps cannot run without a Django project. If there were some way to package the application up as a Python egg (which is the standard unit of packaging in the Python community) and therefore have a very simple install (via setupttools) and deployment (through Paste Deploy), it would be another one of those situations where more features and bugs could be worked out through increased usage. Also instead of using things like a custom type of middleware inside of Django, if Django used WSGI-based middleware as the foundation, we might find that Django users have both access to more tools and user of other frameworks are able to get involved in improving existing ones. In addition, the potential importance of tools like Paste Deploy really need to be analyzed. Its goal is nothing less than to make Python web applications easier to install and run for the end user; to increase adoption of Python on web hosting environments. This is something that is vital to Python as a whole rather than any one framework, and it should be discussed in how Django can support such efforts.

To wrap these suggestions up I must once again say that because Django has such a large audience it also has great responsibility. I hope to see Django move forward continuing to improve upon what exists, continuing to improve upon the already excellent documentation, and most importantly both harnessing the community and not forgetting that there are other projects out there in the Python community with great implementations and ideas; let's try and use them where possible to improve the state of Python programming as a whole. If I sound or have sounded too disparaging of Django in the past, it is because it is something that I have a lot of hope and enthusiasm for, and that I want to see reach its full potential.

All in all, Django is an excellent framework. It is not without some issues, but the focus on documentation and having a well-integrated set of components has led to a framework that has more features than just about anything else out there and is very usable right now.

Pylons

If Zope is on one end of the spectrum from web.py, Pylons is on the other end of the spectrum from Django. The idea of Pylons isn't to provide the user with everything they need, it is to let them chose what they want. Virtually everything can be swapped out, from the URL dispatching mechanism, to the templates, all the way to the ORM - and Pylons helps the user to do so. In some ways it is the roll-it-yourself framework with the hardest things done for you.

Philosophy

What is best for you isn't necessarily best for me. But do you mind if I offer a few suggestions?

URL dispatch

I should preface this by saying that Pylons' method of laying out applications is the classic MVC style. Controllers map to classes and actions map to methods. So my business logic for handling new records would likely be modeled by the class 'RecordController' with actions that mapped to the methods 'search', 'update', whatever. This isn't as simple as Django's mapping to functions, and does not as easily lead to the utilities provided by generic views, but it provides the basic nicities of inheritance; each controller being able to inherit from a master controller which might provide default actions and other functionality.

By default, Pylons uses the Routes URL dispatcher. This system will be very familiar to anyone who has used Ruby on Rails before (it is more or less a direct port). This isn't quite as flexible as pure regular expression matching, but it makes up for it in its basic simplicity and the nice defaults it provides. For example, every Pylons application comes with this route already setup for you:

map.connect(':controller/:action/:id')

This is very nice because it will do virtually all of the basic CRUD style URLs for you without having to specify them. If I have a 'user' controller, for example, with the actions 'new', 'delete', 'edit', I don't have to write a single line of code to map them. It would work the exact same if I added a job controller. Routes however is much more powerful than this, and also supports more advanced features like matching subdomains, matching specific HTTP methods, and even provides a way to generate REST URLs out of the box.

You can ideally switch out the URL dispatching and resolving with something else, and the ability to do so is improving, but this is probably the component that is the most difficult to modify, and that mainly has to do with URL dispatching being so fundamental to the layout of an application.

Templates

Pylons currently by default uses the Myghty templating system. Myghty let's you use regular Python code in your HTML templates. It also has some pretty advanced inheritance features and other functionality. Myghty is modeled after Perl's popular Mason templating system, and as a result has some complicated areas and is not very 'Pythonic' in certain ways. This is why the creator of Myghty, Mike Bayer (the same guy who has done SQLAlchemy, he's a coding machine!), has been working for some time on a successor called Mako. Mako is a good deal simpler and more Pythonic while still retaining the power and functionality from Myghty. It is also one of the fastest templating systems Python has. It wouldn't be inaccurate to consider Mako a better Cheetah.

At some point in the future Mako will be replacing Myghty as the default templating system, but for now Myghty is the default and Pylons' tutorials use Myghty. Using Mako now instead is literally as simple as changing 2 or 3 lines of code. And again, there is nothing stopping you from just as easily usinga different templating library. Pylons is made for the end user to customize it.

ORM

By default Pylons doesn't really come with an ORM. You can enable SQLObject by uncommenting a few lines of code, but most of the tutorials use SQLALchemy, and it is the suggested ORM and likely will be the main ORM Pylons focuses on in the future. SQLObject is roughly equivalent to Django's ORM, but SQLAlchemy is extremely powerful. It gets this power from not doing a simple mapping of tables to classes and records to instances, but in rather allowing the user to map objects to parts of the database themselves, including items like relations. There isn't too much SQLAlchemy cannot do, unfortunately this power comes at a price. SQLAlchemy isn't the simplest database system to understand or use, and it has several ways of doing things. You can use the lower-level functionality and do the mappings yourself, or you can use slightly higher level functionality and let SQLAlchemy do much of the mapping for you (assignmapper). Finally, if you really want something that feels very Pythonic and easy to use like Django's ORM, there is the recent project Elixir.

This proliferation of possibilities is unfortunately a slight problem in the Pylons community. Different tutorials use different ways of doing things and if you want to use SQLAlchemy in your project there's a bit of manual setup involved which is different depending on what exact method you are using. This is pretty difficult on the new users, but once you've decided upon what you want, the Pylons user has a lot of power in maniuplating their database.

Form handling

Pylons currently uses the WebHelpers library to help to generate small pieces of HTML, including form items. These are based on a port of Ruby on Rails' helpers, and they work fairly well as well as being easy for the user to write their own (they are after all only simple Python functions). The following piece of code is an example of generating a text box.

>>> text_area("body", '', size="25x10")
<textarea name="body" id="body" cols="25" rows="10"></textarea>

Pylons uses Ian Bicking's FormEncode to do validation of HTML forms as well as the conversion of text input into their correct Python values. You specify a schema for your form (this feels similar to how Django's newforms works), and them FormEncode knows what to expect. FormEncode doesn't, however, generate your form for you. That is up to either you to do manually or to use a tool like FormBuild. In the future, however, Pylons will be using Toscawidgets, which is Turbogears widgets system for 2.0 that is designed to work with multiple frameworks. These widgets will allow you to put the logic of display and validation of forms and other HTML elements into their own small bundles. It also will deal gracefully with AJAX. A good way of looking at Toscawidgets is as a slightly more powerful version of Django's newforms system (newforms is designed only to handle forms while Toscawidgets has broader objectives).

The main problem with this, however, is that Toscawidgets isn't currently really well documented or heavily used. Therefore the user is stuck for the most part manually building forms, which might be terribly inconvenient depending on your viewpoint.

Extras

WSGI WSGI WSGI. Pylons is focused around WSGI. Where as some frameworks just use WSGI at the bottom of their stack to interface with the server, Pylons uses WSGI throughout, and this is exactly what allows for Pylons' great modularity and flexibility.

The real power of Pylons comes from its foundations on another project by Ian Bicking, Paste. Paste includes small WSGI middlewares and libraries which would help any framework to handle things like file uploads, error handling, request and response objects, and several other things (there's even a pony?). Perhaps more important than these are two other components: Paste Deploy and Paste Script. As I mentioned earlier, Paste Deploy is designed to make it easy for end users to install and run any Python application. Paste Deploy uses basic .ini files for configuration and standard Python eggs and easy-install as the method of installation. Since every Pylons app is an egg, it is simple to both upload one of your projects to the Cheese Shop as well as to download, install them, and then run them on your own machine (if you've never used a Python egg, it can literally be as easy as typing 'easy-install Pylonsapp' at the command line). Pylons doesn't try to create its own application system like Django, it uses what already exists in the Python community and I find that this level of reusability is both brilliant and unmatched in any other framework. Driving all of this is Paste Script. Paste Script provides basic command line commands (what else?) that help you to administer your application. For example, to run an existing Pylons application, you would enter:

paster server --reload development.ini

To create a completely new application, you'd enter:

paster create myproject --template=pylons

In addition Paste Script makes it fairly easy to add your own commands. One thing I'd like you to notice in that last example is the inclusion of the 'template' argument. Several default templates exist such as 'pylons' and 'pylons-minimal', but you can also create your own templates. So if my company has a specific header logo and CSS layout that is needed to be in every single project, I could modify the base 'pylons' template and put my own custom layout in there. Then whenever I needed a new company app, I could simply specify 'customtemplate' as my template instead of the regular 'pylons' one. This also allows me to make whatever modifications I want to the default Pylons setup such as using a different ORM, using a different directory layout, using specific settings, and then being able to easily reuse that template later on. Really the possibilities with this are endless, and Toscawidgets even uses it to generate new widgets for the user to modify.

Advantages

Pylons is really made to be what you want it to be. If you ever find that a certain templating system just doesn't fit your style, or it is buggy, or perhaps that there is simply a better one, just change a few lines of code and you are good to go. Pylons offers you some suggestions but it makes it easy for you to override them. This also makes Pylons very future-proof. If some new component comes out and blows the socks off everyone, integrating it into Pylons will not be difficult. In addition the developers of Pylons are able to focus on the minimal parts of the framework itself rather than larger components like ORMs, the interfaces and basic needs, which allows the codebase to be very slim and for different developers to do what they do best. Like Ben Bangert said the other day in IRC, "I'm not a template developer, you'd have to ask...". While Ben might not be a expert at designing powerful ORM systems, Mike Bayer is, and it allows everyone to focus on what they do best instead of having a project which tries to do everything at once and inevitably ends up doing many things well but not many anything great. In this regard Pylons reminds me a lot of the ideology behind Unix and Unix-based systems.

Add to this the fact that the project leaders are almost always immediately accessible in IRC or via email, and are constantly fixing bugs and making improvements, and you've got a development process which just seems much more accessible than Django or Rails.

One more thing: if you are moving over to Python from using Rails, Pylons will probably be the framework that is most immediately intuitive to you, as it has borrowed many of the same concepts.

Disadvantages

Unfortunately, it is this very modularity and flexibility which can cause problems. It is not Pylons' job to document every ORM that can be used with Pylons. Nor is it that job of Pylons to document templating systems. Neither is it that responsibility of the framework authors to fix bugs or improve the functionality of certain components. They may decide to do so, but there are no guarantees this may happen. As a result if you are using as an example SQLObject with Pylons and you run across a bug, you first must figure out if it is a bug related to Pylons or a bug related to SQLObject, and then you must submit that bug to the correct place and hope they fix it. Luckily this isn't too much of an issue because the authors of the major projects used in Pylons (Paste, SQLAlchemy, Mako), tend to be active members of the Pylons community, but it is always a potential problem.

James Bennett did a good job of describing some of the potential problems in a 'glue' framework in this post from February

Documentation is also somewhat of an issue in the Pylons community. Because again, Pylons cannot document every component, you are on your own hoping that documentation for a component is pretty good (luckly it generally is), and the documentation for one component will almost certainly be in a different style than another, not to mention there is no single location to find it all. In addition, because there are multiple ways of doing things in Pylons, one tutorial might do things one way and another tutorial another. There is less consenus on what the 'right' way of accomplishing things is. And because there is less consensus, if you come across some corner case, there is no guarantee that anyone has pioneered the way before you and made any kind of effort to document the solution.

Topping this off, some cutting edge components like Toscawidgets are currently just out of reach of most people because either they aren't totally ready or they don't have documentation. For Pylons users there is a bit of a paradox: they are both able to use the latest stuff in the Python community, and in some ways limited to the progress of the community at the current time.

For this reason Pylons can be a bit more difficult for new users to get into with its smaller community and less thorough documentation than Django. Pylons doesn't do much hand-holding. There is an assumption that you have some idea of what you are doing. It is always improving, though, so hopefully this well be less of an issue in a few months.

Future and concerns

What exactly composes Pylons tends to be changing quite a bit. In a few months Mako will likely replace Myghty as the default template engine and hopefully Toscawidgets will be ready for production use inside Pylons.

My main concern with Pylons doesn't have to do so much with the framework itself. It is fantastic piece of technology and the ideas behind Pylons (WSGI, Paste Script, and Paste Deploy) are things that I feel can genuinely revolutionize web development.

Unfortunately, as I stated earlier, documentation is not to the level of Django's. Pylons is not nearly as accessible to a newbie to web development. What I'm really hoping to see is for Pylons to include SQLAlchemy as the default ORM and to decide upon a general way of using the ORM, whether that is through assign_mapper or Elixir, or whatever; this default needs to be in the tutorials and what is the primary suggestion to any users. Further, defaults should be chosen for every aspect of web development. Then those defaults and how they work inside of the framework could be heavily documented and focused on to provide a framework which is as user-friendly as Django but provides flexibility that is unparalleled. I have been told before that this is 'not the Pylons way', but there are some suggestions within the Pylons community for something like this. As an extra, it would be great if Pylons could provide functionality similar to what Django does with some type of admin application, something for user registration, an application for comments, etc. This too might be outside the scope of Pylons, but there are efforts underway to do things similar to this, and I've heard it stated before that an eventual paster template for Pylons may include AuthKit for user registration/authentication.

Whether or not Pylons is a long-term 'success' long-term depends upon if developers really buy into the idea of standards and start to develop applications and middleware in a way that makes them reusable by multiple frameworks, and installable through things like eggs. If none of this ever catches on, a lot of the potential of something like Paste Deploy may be wasted. It is more work to make things standard compliant and flexible, but it is ultimately what can really change things.

TurboGears

And at last here we are with TurboGears. TurboGears was started in 2005 by Kevin Dangoor somewhat in reaction to Rails. The entire thought process behind TurboGears was to use existing libraries and components in the Python community to create a 'megaframework' which could rival Rails in functionality and prestige. It was considered a 'best of breed' framework, and it was believed that by using existing components TurboGears would get the benefit of these components being used and improved outside of the TurboGears community. The framework did not initially start out to be flexible, but it was quickly realized that what is 'best' is both subjective and temporary, and TurboGear's renewed focus on flexibility is one of the things which has greatly influenced Pylons and its development.

Philosophy

We'll provide you with the best components available, but we won't make it hard for you to use something else.

URL dispatch

URL dispatching is arguably what defines TurboGears most. It is based around the CherryPy framework, which maps classes and methods directly to URLs and arguments to those methods directly to query args.

So for example, the URL '/user/add/?name=greg' would map to the following bit of code:

class User(object):
    @expose
    def add(self, name='default name'):
        pass

Now I will be the first to say that this isn't a completely accurate representation of how TurboGears maps URLs, but it is a pretty good representation. The great thing about this is it is ridiculously simple. It is Python all the way through. Just classes, methods, and arguments. You don't have to worry about concepts such as controllers, you don't have to write regular expressions, and for the most part you don't have to think too much about the fact that you are writing a web application.

The problem with this beautiful simplicity is that it simply isn't that flexible. Your URLs are defined by your application layout, which is not very realistic, isn't very easy to change, and often does not work well for legacy applications. So while this choice was loved by some, it was hated by others. There have been those that have used Routes with CherryPy, and the latest version of CherryPy, version 3.0, has functionality to allow you to change how URLs are dispatched. Regardless, this design decision still very much is a part of TurboGear's character.

Templates

TurboGears made another interested choice when it came to templating. Instead of chosing a text-oriented system like Django or a Python-oriented system like Myghty, TurboGears used the XML-based Kid. Looping, method calls, and other Pythonic constructs were added via the XML namespace prefix py. Here is a sample piece of code:

<p py:if="5 * 5 == 25">
    Python seems to be handling multiplication okay.
</p>

This gave Kid some interesting characteristics. First of all, every TurboGears template is a well-formed XML document, which is very nice. In addition it allows the use of XML-based tools for processing. On the other hand, this reliance on XML also made Kid somewhat unsuitable for pure text files such as CSV, and added a lot of overhead to the point where Kid templating is fairly slow.

Due to some of these issues with Kid, a successor what developed, named Genshi. It alleviated several of Kid's problems and also added support for XPath and XQuery. Genshi still isn't the fastest templating language out there, but it is a solid one and for those who enjoy XML-based templating it is very powerful. XML isn't the most beloved of technologies, however, so this XML-based templated added another level to either users loving TG or hating it.

ORM

I'd be duplicating too much info if I covered it here, basically TG's ORM situation is very similar to Pylons. TG started out using SQLObject and since then has moved to using SQLAlchemy.

Form handling

You can thank TurboGears for the wonderful newforms library that you can find today in Django; in many ways Django's component is very similar to the TurboGears widget implementation designed more than a year ago. TurboGears' widget system encapusalted the logic and display for small pieces of HTML, CSS, and AJAX and used FormEncode for form validation. Unfortunately the widget system had some issues that were only discovered after heavy use, and they were also tied heavily to both Kid and CherryPy. This is what led to Toscawidgets which is a templating language and framework neutral reimplementation of TG's widgets that will be available in TurboGears 2.0. For now when using TG 1.0, however, the older system still works fairly well and is decently documented.

Extras

TurboGears has done a great job of providing all types of functionality. TG unlike many other frameworks comes bundled with AJAX functionality through the Mochikit JavaScript library. It also includes a widgets browser, admi18n which allows the user to localize their application, ModelDesigner and Catwalk, tools which help with creating and editing database models, Info, a tool which lists installed packages used with TurboGears, and a web-based console. Basically a ton of neat items which help the programmer to get their work done faster and more efficiently.

Advantages

I just listed many of them. TurboGears comes with a lot of administration tools that other frameworks do not have. In addition there has been more than a year of work on tying the current components together and the documentation is fairly good. TurboGears 1.0 is a mature framework. TurboGears also does a good job of allowing you to swap out templating languages and other pieces, in this way not entirely unlike Pylons.

Disadvantages, future, and concerns

I'm cheating a bit here and going to merge these two sections into one, because I feel like in TurboGears they are closely related.

The main problems with TurboGears is it has been realized that there are issues with some of the functionality in 1.0 which is why 2.0 is being created and several key components are being swapped out or upgraded; SQLObject to SQLAlchemy, Kid to Genshi, CherryPy 2 to CherryPy 3, and a few other changes. Essentially, 1.0 is a dead framework or the 'old and busted' components that nobody really wants to use, but it is the only framework available right now. Most of the documentation is based upon 1.0, quite a few of the admin tools such as Catwalk simply won't work with 2.0 without a rewrite, etc. This has caused some real problems in the TG community. People don't really know whether to stick around using and working on improving 1.0 and the documentation, or if they should wait for 2.0. Added to these two options is the third, less complete one to manually swap out components from that will be used 2.0 with those in 1.0, and this is what many have done. This level of confusion has caused the community to slowly decline.

TurboGears was and is in a really difficult spot. It was almost ahead of its time, and since it was based around using the 'best of breed' libraries in the Python community it had to make these major changes. Only shortly after TurboGears really started to get popular, SQLAlchemy became a real alternative to SQLObject, and there were many cries within the community to use it instead. In addition, WSGI which had been somewhat of a secondary concept in Python soon became extremely important with the emergence of Pylons. The TG developers were faced with two options: either rework the framework from the ground up to include these newer components or to stick with what already existed and soon become outdated and an afterthought. The developers did the correct thing as difficult as it was and decided to rework the framework to integrate these newer libraries. Unfortunately it has led to the aforementioned confusion. Heck, even the TurboGears book released in November 2006 is largely outdated.

The focus of 2.0 more than anything else is modularity. CherryPy 3 is much more WSGI-based than its predecessor and just about every component available is being broken down into decoupled packages (such as the Toscawidgets library). TG will also be using Paste Deploy and other Paste-related tools further in its stack. Infact, TurboGears 2.0 has many of the same goals and overall design as Pylons, and therefore there has been some talk of the two projects merging. The notable difference between the two frameworks, however, is TurboGear's reliance on CherryPy as its core, and this is not something easily reconciled with Pylon's design. Ian Bicking has offered a good comparison of the similarities and differences between the two frameworks at this link.

This might not be an accurate view of the project, but I think TurboGears is in a lot of trouble. As I've mentioned, the community has declined a lot over the past year. Users have shifted over to both Pylons and Django. The maling lists tend to be pretty quiet, and the last time I viewed one of them it had quite a bit of spam. Go into the IRC room and ask a question and you'll be lucky to get an answer within the hour if you get one at all. In the end, even if 2.0 ends up being an amazing project (and I can't imagine it being implemented and well-documented in any time shorter than a year), first impressions are key, and many people still have an image of TurboGears as the frankenstein framework which couldn't quite figure out what it wanted to be. How will TG possibly remarket itself?

This is all very unfortunate. TurboGears has gone from being perhaps the most discussed framework at the end of 2005 to one where the community is much smaller than Django's and which even Pylons has eclipsed. It is this combination of bad circumstances which leads me to believe that a merger between Pylons and TG would be the ideal situation if at all possible. If the main thing that separates the two projects is the reliance on Paste by Pylons and the reliance on CherryPy by TG, whatever steps possible should be taken to see if the frameworks can work together. However, it should be noted and has been stated by others that becuase the focus in both Pylons and TurboGears is modularity, the existence of one is not negative to the existence of the other. An admin application written for Pylons should theoretically be able to plug right into TurboGears 2.0 and vice versa. Despite this, it is with some sadness that I must say I have a very difficult time recommending TurboGears to a new web developer, though this could change in the future. There simply isn't enough stability right now. Nobody should have to spend a few months learning the set of tools available in 1.0 and then have to drop them and use the new ones for 2.0.

Wrapping it up

This has taken a lot more time and writing than I had intended. I hope this has been a thorough and fair analysis, and I hope this has perhaps helped you in your choice of frameworks. In the end a developer can only pick what seems best for their situation; nobody can tell you what is best and there is no 'one size fits all' solution. The two main projects that I would recommend are either Django or Pylons, and you would be well-served to try either one. Actually, you really should try them both. Pick a small project or a couple of projects and spend some time learning and using each one. Neither are terribly complicated and both have a friendly community that will help you along. When you get down to it, both have different design goals; one project focusing on being modular and flexible (Pylons), the other being a single set of well-integrated and documented components (Django). Ultimately the existence of these two frameworks and ideologies can only lead to further innovation in Python web development, and it is my hope that both frameworks take the time to learn from each other. And even if TurboGears is in a slump that it never recovers from, it should be remembered in the Python community because it has had a great influence on many projects.

last updated 2 years ago #