Retro is a lightweight Python Web WSGI-based toolkit that is designed for easy prototyping and development of Web services and Web apps.
Some facts:
- Retro is notably one of the very few web toolkit able to do HTTP streaming (aka. Comet) out of the box.
- Retro comes with batteries included and features a simple, fast, reactor-based WSGI server.
- Retro makes use of Python decorators for declarative web development.
Core concept: web services are "just" message-passing over HTTP. Regular Python classes (and their methods) methods can be turned into web services using only decorators.
1.Seeing is Believing
Retro is for hackers, so here is the code.
1.1.The Telephone: A Basic Web Service
<<< from retro import ajax, on, run, Component
class Telephone(Component):
def _init( self ): Component.init_(self) self.tube = []
@ajax(GET="/listen") def listen( self ): if self.tube: m = self.tube[0] ; del self.tube[0] return m
@ajax(GET="/say/{something:rest}") def say( self, something ): self.tube.append(something)
run(components=Telephone()) >>>
run this script with python
:
<<< $ python telephone.py Dispatcher: @on GET /listen Dispatcher: @on POST /channels:burst Dispatcher: @on GET /say/{something:rest} Retro embedded server listening on 0.0.0.0:8000 >>>
and then interact with your new web service using curl
curl http://localhost:8000/say/hello null curl http://localhost:8000/say/world null curl http://localhost:8000/listen "hello" curl http://localhost:8000/listen "world"
1.2.The Watch: A less basic Web Service
Now "The Telephone" was rather trivial. Let's do something more impressive : a web page that displays the time.
from retro import ajax, on, run, Component import time
<<< class Watch(Component):
@on(GET="/time") def getTime( self, request ): def stream(): while True: yield "
%s" % (time.ctime()) time.sleep(1) return request.respondMultiple(stream())
run(components=Watch()) >>>
Now start firefox and go to http://localhost:8000/time (you won't be able to open two tabs with it, if you want to test concurrency, use another browser or another machine).
1.3.The Chat: A less basic Web Service
The Watch was pretty interesting, but the time.sleep(1)
in the middle of the generator doesn't really help performance. Let's do some more fancy stuff.
2.Reference
2.1.Features
- Based on WSGI
- Embedded reactor-based WSGI server
- Flup for FCGI/SCGI/AJP connection
- Plain old CGI mode
- "Comet"/HTTP streaming support
- Decorators for declarative web services
- Lightweight
2.2.Architecture
APPLICATION ----------------- COMPONENT ----------- -------- COMPONENT>
2.3.Dispatcher
To be able to server pages and content in your application, you have to express a mapping between URL and actual methods of your components.
In Retro, you define a mapping by exposing a method of your component using the @on
decorator:
class Main(Component): ... @on(GET="/index") def index( self, request ): return request.respond("Hello, world !")
Here, we've defined the 'index' method (the name is not important), and exposed to react to a GET
HTTP method sent to the /index
URL. Restarting your server and going to http://localhost:8080/index will give you this text:
Hello, world !
The @on
decorator parameters are made of the following elements:
- a parameter name (like
GET
,POST
,GET_POST
), where the HTTP methods are uppercase and joined by underscores
- a parameter value, which is an expression that defines matching URLs.
- an optional
priority
, which allows one mapping with higher priority to be used in preference when more than one mapping matches the URL.
The parameter value expression can contain specific parts that will be matched and given as arguments to the decorated method:
@on(GET="/int/{i:integer}") def getInteger(self, request, i): return request.respond("Here is number %d" % (i))
Now, if you go to http://localhost:8080/int/0, or http://localhost:8080/int/1, http://localhost:8080/int/2, you will see these numbers printed out.
Generally speaking, anything between {
and }
in parameter expressions will be interpreted as a matching argument. The format is like that:
{ NAME : EXPRESSION }
where NAME
must match an argument of the decorated method, and where EXPRESSION
is either a regular expression (as of Python re
module), or one of the following values:
word
,alpha
: any sequence of alphabetical charsstring
: everything that is not a/
digits
: a sequence of digits, cast to anint
number
: a floating point or an int, negative or positiveint
,integer
: an int, negative or positivefloat
: an float, negative or positivefile
: two alphanumeric words joined by a dotchunk
: everything which is neither a/
nor a.
rest
,any
: the rest of the URLrange
: two integers joined by:
(Python-style)
Once you have your mapping right, you may want to do more complicated things with your exposed methods… and we'll see in the next section what we can do with the mysterious 'request' parameter.
2.4.The Request object
The retro.core.Request
class defines a class that represents an HTTP request. The request
parameter we've seen in the previous section is an instance of this class, that represents the request sent by the client browser and received by the Retro server.
The request object offers different kind of functionalities:
- Accessing the request parameters, cookies and data: whether
POST
orGET
, whether url-encoded or form-encoded, parameters and attachments are retrievable using theparam
,cookies
anddata
methods.
- Accessing request method, headers and various information: TODO
- Creating a response: the request object contains methods to create specific responses, whether it is serving a local file, returning JSON data, redirecting, returning an error, or simply returning specific content. These methods are mainly
respond
,returns
,redirect
,bounce
,notFound
,localFile
.
In practice, you need only to know a few things. First, how to get access to parameters.
Say you have a request to that URL:
http://localhost:8080/api/doThis?a=1&b=2
and that you have a handler bound to /api/doThis
, here is how you would get access to a and b:
@on(GET_POST='/api/doThis') def doThis( self, request ): a = request.get('a') b = request.get('b') if a is None or b is None: return request.respond( "You must give proper 'a' and 'b' parameters", status=400 ) else: return request.respond( "Here a:%s and here is b:%s" % (a,b) )
This is the simplest, and most common case. Now if you want to receive a file that was POST
ed by the client, by a form like this one:
<form action="/api/postThis" method="POST"> <input type="text" name="name" value="File name" /> <input type="file" name="file" value="Upload file" /> </form>
you would do the following:
@on(POST='/api/postThis') def postThis( self, request ): file_name = request.get('name') file_data = request.get('file') file_info = request.file('file') return request.respond( "Received file %s, of %s bytes, with content type:%s" % (file_info.get("filename"),len(file_data),file_info.get("contentType")) )
we see here that the file data is available as any other parameter, but that we can use the request.file
method to get more information about the file, like its content type and original filename.
request.[respond, returns, redirect, bounce, ...]
methods. These methods
actually return a generator that will be used by Retro to produce the
content of the response.3.Deployment
3.1.Standalone
Retro comes with its own WSGI server, which is the more well-tested solution for deploying applications that make use of advanced WSGI features (such as request streaming using yield
.
Usually, a standalone Retro application ends with code similiar to this one (here we have an application composed of two FileServer
and PageServer
custom components).
def createApp(): return Application(components=( FileServer(), PageServer(), )) def start( app=None, options=None ): if app is None: app = createApp() name = "My application" run( app=app, name=name, method=STANDALONE, port=options.get("port") or 8888, sessions=False, ) if __name__ == "__main__": options = {} for a in sys.argv[1:]: a=a.split("=",1) if len(a) == 1: v=True else: v=a[1];a=a[0] options[a.lower()] = v start(options=options)
If your main server script is server.py
, you can simply start it like this:
python server.py PORT=8888 Dispatcher: @on GET /crossdomain.xml Dispatcher: @on GET /lib/css/{css:[\w\-_]+\.css} Dispatcher: @on GET /lib/swf/{script:\w+\.swf} Dispatcher: @on GET /lib/images/{image:[\w\-_]+\.(png|gif|jpg)} ... Retro embedded server listening on 0.0.0.0:8888
The next thing is to make this application available to your website, meaning you have to map it to port 80
. You can bind it directly to port 80
, but it's a better idea to use proxying.
Here is how to do it with Apache:
ProxyPass /myapp http://localhost:8888/ ProxyPassReverse /myapp http://localhost:8888/ ProxyPreserveHost On
You can also use something like Pound, where a configuration would look like this:
ListenHTTP Address localhost Port 80 # Your Retro application Service URL "/myapp/.*" BackEnd Address 0.0.0.0 Port 8888 End End End
Pound is definitely cooler, as the configuration is easy and it's a very fast reverse proxy. What you can do is use Apache, Lighttpd or Nginx to serve static files (like '/lib/.*'), overriding your Retro application default server for static files.
3.2.With FastCGI
Retro can be connected to an FCGI (FastCGI) enabled server thanks to the Python FLUP module. Here is how to do it using Lighttpd:
First create a myapp.fcgi
script:
#!/usr/bin/env python from retro import * from retro.contrib.localfiles import LocalFiles from myapp import server print "Retro: Starting FCGI" run( app = server.createApp() name = os.path.splitext(os.path.basename(__file__))[1], method = FCGI, sessions = False ) # EOF
The above script expects you to have a myapp.server.createApp()
function that returns the retro application.
Second, create a myapp.lighttpd.conf
script:
server.modules += ( "mod_fastcgi" ) server.document-root = "." server.port = 8888 fastcgi.debug = 1 fastcgi.server = ( ".fcgi" => ( "localhost" => ( "min-procs" => 2, "socket" => "/tmp/retro-fastcgi.socket", "bin-path" => "/full/path/to/myapp.fcgi", ) ) )
Notice that you have to give the full absolute path to myapp.fcgi
in your bin-path
field of the lighttpd configuration file.
Now type the following command:
$ lighttpd -D myapp.lighttpd.conf 2009-04-02 14:29:02: (log.c.75) server started 2009-04-02 14:29:02: (mod_fastcgi.c.1325) --- fastcgi spawning local proc: /home/sebastien/Projects/Public/Retro/Examples/FCGI/myapp.fcgi port:0 socket /tmp/retro-fastcgi.socket min-procs: 4 max-procs: 4 2009-04-02 14:29:02: (mod_fastcgi.c.1350) --- fastcgi spawning port: 0 socket /tmp/retro-fastcgi.socket current: 0 / 4 ...
And now open your browser and go to http://localhost:8888/myapp.fcgi/. You can now play with Ligghttpd configuration options to change the root URL for your application.
Note that Lighttpd FCGI configuration is quite painful, as it's very hard to know what doesn't work when things don't work. I'd recommend to use Apache's modwsgi or a combination of Retro standalone and 'pound' load balancer.
3.3.With modwsgi
Using modwsgifor Apache is by far the easiest solution to get a Retro application up and running:
WSGIScriptAlias /altitude /home/sebastien/Servers/Datalicious.ca/altitude.wsgi
#!/usr/bin/python import sys, os sys.stdout = sys.stderr import myapplication.server application = myapplication.server.createApp() # EOF
And some tips:
- In your startup script extend and manipulate sys.path to make sure you control the environment properly.
- In your startup script, change the current working directory to make sure you run at the proper location
- WSGI is by default very similiar to CGI (one process per request), but with more work in configuration you can get it work like FCGI.