Python Web Application Deployment Frustrations

killall -USR1 dispatch.fcgi

This will restart all the fcgi processes the next time they're called, so it'll be slow on the first call after this, then quick again. In theory this shouldn't disturb anyone using the site.

killall -9 dispatch.fcgi

This will force all the processes to immediately exit, potentially dropping sessions, so try doing a tail on the logs first to check no one is using the site.

These are for rails on dreamhost but I imagine the same works for python fcgi. Perhaps you could store your sessions in a database table instead?

# SESSIONS

sessionFilter.on = True

sessionFilter.storageType="File"

sessionFilter.storagePath="/apps/session"

sessionFilter.storageFileDir="/apps/session"

I'm not sure whether I need both "storagePath" and "storageFileDir", but I haven't noticed any problems since configuring it like this.

sessionFilter.on=True

sessionFilter.storageType="File"

sessionFilter.storagePath="(my home dir)/app_deploy/session/"

I don't think you need both. This resolved the disappearing reappearing problem, but I still have the session deadlock problem.

This talks about mod_fastcgi and sockets. Sounds ugly.
http://rfc1437.de/page/django-apache-and-fcgi/

Maybe these instructions using mod_scgi instead will help.
http://simon.bofh.ms/cgi-bin/trac-django-projects.cgi/wiki/DjangoScgi

Will your host let you run lighttpd and have Apache be only a proxy? Then this will help:
http://www.cleverdevil.org/computing/34/deploying-turbogears-with-lighttpd-and-scgi

Rick Thomas

As for the deadlock error, I haven't seen that in a long time on my server. I can't really say I know what fixed it, but I don't see it anymore. Have you tried deleting all your session files? (Not that any of this *should* be necessary...)

I never deleted my session files in order to get it to unlock. I got to where I always deleted them after I had killled all the fastcgi processes.

Your situation is something that terrifies me. It's the same issue I had with the random "NoneType is not callable" with Kid templates and it's the same issue I'm having now with the deadlock problem. It's not reproducible. And it's random. And it appears to clear up for a while, then all of a sudden ... bam! You have a randomly occurring bad behavior on your hands. That's much worse than something crashing in a repeatable fashion. That's something I can't take a chance on.

So, for the boring technical details you don't want :-), the product catalog I've mentioned contains general designs that she's worked upand any example images she wants to tie to each design. The shopping cart (admittedly overkill for this site, but, hey..) allows customers to drop in a design with custom information such as the baby's birth date, time, weight, etc - stuff you'd want on a birth announcement. The payment system passes the customer off to PayPal so they can handle credit card processing - 'cause that's something I don't want to mess with! And I've already begun working on phase 2, which is integrating this site into her CafePress store(s).

Back to the non-technical details, at CafePress, she has some designs that go along with the birth theme that she's selling on TShirts, buttons, magnets, etc.

Hope that wasn't too much information! Thanks for asking!

I am now in a similar situation as you were beginning this year. I am trying to deploy my TurboGears app on DreamHost. Everything worked out as advertised on the TG Wiki, and I was hilarious to see my app running.

And then the misery began.

If you don't interact with the webapp for a while, it takes a lot of time to load. More than a minute. Unacceptable. I saw that the server starts up actually. And it's not only if you don't interact with it. It happened also after just a few minutes playing around.

It's generally very slow to respond. I didn't profile to see where the bottlenecks are, but even for simple operations - one query with few records in the database - takes several seconds to execute.

The app has a lot of redirects: part of it is implemented using command pattern, so eventually all commands get redirected to an appropriate URL. After the command finishes, it redirects back to the main page. Bad idea. Redirects don't really work always in this set up. They make it slower. I think that the problem is that they go through the FastCGI instead of being resolved by cherrypy.

And sessions - I had similar problems with them.

I am considering switching to Django, so I will welcome your feedback. I also thought of RoR, but I am afraid of the learning curve. I know it would be fun, but I think it would just take too long, especially that I only have theoretical knowledge of Ruby (and I have been programming in Python for a while).

First, the first stage of the Django deployment is complete. We have some more features in the works, so it's by no means a "done site". The process of deployment was a snap. But then again, so was the TG app. I am happy to say that I am having no problems, neither the problems I had with TG, nor new problems cropping up. The speed is quite good, I think. As you mention, FastCGI does take some time to come back up if idle for a while, but I've mostly alleviated that with a simple checker script that runs on my server at home and hits a test URL on the app. You can see for yourself how it performs by going to http://pitterpatprints.com/. The pages are slow to load the first time due to the heavily graphical nature of the site, but after that, it's pretty snappy. http://pitterpatprints.com/store/products/ and http://pitterpatprints.com/store/product/some_product_id/ are almost totally database driven, so that will show you something about DB response time.

Hope this helps. Feel free to post here with questions or email me directly. You should be able to find my email address on my author's page or on the main page of this blog entry.

I will try the trick with a crontab to keep my FastCGI alive.

There is still some brushing up to do with these frameworks - you still need to jump through a lot of loops to get the basic things running smoothly.

How does the deployment work with Django - do you also have to kill of the FastCGI processes for it to pick up your changes?

For Django, you do have to kill the fastcgi processes in order for them to pick up your changes. But, you should be able to issue a ``killall -USR1 your_fastcgi_process_here`` and it doesn't totally kill all of them. It just makes them aware that they need to reload when the next request comes in. At least that's how I understand it.

This is one of the reasons why I like web applications - that all your users have one and only one version of your code, and that you can easily deploy new versions any time.

Plus Spyce has some of the best features of JSP (custom tags) and ASP (active handlers) plus some unique features of its own. I daresay many people might find MVC to be a turn-off after trying out Spyce's facilities.

http://en.wikipedia.org/wiki/Spyce
http://spyce.sf.net