Real-Time Web: Gevent and Socket.io (Rick Copeland)

Brief survey of asynchronous (defined as an event loop for purposes of this talk) programming

• AsynCore
  • In stdlib, used for stdlib SMTP server
  • Nobody cares about it anymore
• Twisted
  • Everybody knows this is the standby for asynchronous Python
  • Callbacks hurt my brain, too, assuming you mean nested more than one or two levels
• Stackless
  • lightweight threads for cooperative multithreading
  • Needs a custom Python, though :-(
• Event-based green threads
  • Like Stackless, but in regular Python
  • You know when you yield.

Gevent

• Gevent is an example of the last option.
• Unsurprisingly, you should use it for I/O thanks to the GIL.
• spawn_link means you’re linking to a spawned greenlet?
• Greenlet provides the green threads.
• Sweet, glorious timeouts! Timeouts suck HARD for “real” threads.
• Pool implementation; it’s so nice when langs/libs give you these abstractions instead of rolling your own.
• Event looks like fairly low-level primitives, but also multiple Queue implementations. On a side note, why does everybody love talking about LIFO queues instead of Stacks.
• The big use-case is networking. They give you “green” versions of sockets, select(), SSL, DNS, etc.
• Good WSGI support:
  • gevent.wsgi wraps a C lib, doesn’t do streaming, pipelining, or SSL
  • gevent.pywsgi is a little slower (does 75% of the reqs/s) but is full-featured.

The “Real-Time Web”

• No page refreshes
• Server push
• Examples: Chat, analytics, etc.
• Implementations: Flash, polling, Web Sockets, and more!
• Socket.io handles papering over all the different implementations and picking the best one based on the browser, etc.
• There’s a Gevent library to give you Socket.io support (nice!)

ZeroMQ (wonder how this fits in)

• I watched Zed’s talk; ZeroMQ is cool.

Refresher

• ZeroMQ “sockets” are message-based, delivery is via a dedicated communication thread.
• Supports TCP, inproc, unix, multicast transports
• Different types of sockets: REQ/RES, PUSH/PULL, PUB/SUB, et al
• pyzmq is great, but you need gevent_zmq for Gevent
• Be careful when forking or otherwise using multiprocessing; Gevent and ZeroMQ both have global(ish) things.
• VERY NICE! He’s using the module pattern in his sample JS. Hooray for high-quality sample code.

Miscellaneous

• The slides for this talk are up on SlideShare.

Cython: Pythonesque C Extensions (Matt Hagy)

• You need a .pyx file.
• You can use this to get a huge perf win by using Cython, but you can also use it to just make it easier to wrap existing C libraries.
• It’s almost a superset of Python (I wonder what prevents it from being a proper superset).
• Special features for working with NumPy arrays.
• Neat: a language-level feature for thread-level parallelism. Since it’s C, you don’t have to hold the GIL.

Writing Acceptance Tests with Freshen (Doug Hellmann)

Freshen

• Freshen is based on Cucumber from Ruby, similar to Lettuce.
• It’s GPL

Defining Tests

• Features - they group Scenarios
• Scenarios
  • Use Given to declare preconditions
  • Use Then to declare expected outcomes
• Supporting Code - applies the test definitions to the application under test
• You can use a “Background” to declare your “globals”
• Cool feature: Python’s multi-line strings
• The process of applying the tests to your system is interesting. I could explore this for quite a while, I think. Maybe I should show it to the QA guy at work.
• The actual tests run inside of nose as a plugin. It detects .feature files as “Features” and processes them accordingly.
• Parameterized testing are handled via “Scenario Outlines” (I hope it works well, because parameterized tests are a total pain in JUnit).
• It is very new (only on v0.2), so you probably want to run with the version from GitHub.

From the Q&A

• The error-handling, etc., seems to be much better than Lettuce, hence going with it instead of Lettuce.

Brandon Writes a Program… LIVE!

• PyEphem is a module that Brandon maintains and sounds cool for astronomy buffs.
• Astronomy is fun.