The Home for People Who Like to Flip Out and Write Code
Notes From PyATL 2011-11-10
Real-Time Web: Gevent and Socket.io (Rick Copeland)
Brief survey of asynchronous (defined as an event loop for purposes of this talk) programming
- In stdlib, used for stdlib SMTP server
- Nobody cares about it anymore
- Everybody knows this is the standby for asynchronous Python
- Callbacks hurt my brain, too, assuming you mean nested more than one or
- lightweight threads for cooperative multithreading
- Needs a custom Python, though :-(
- Event-based green threads
- Like Stackless, but in regular Python
- You know when you yield.
- Gevent is an example of the last option.
- Unsurprisingly, you should use it for I/O thanks to the
spawn_link means you’re linking to a spawned greenlet?
Greenlet provides the green threads.
- Sweet, glorious timeouts! Timeouts suck HARD for “real” threads.
Pool implementation; it’s so nice when langs/libs give you these
abstractions instead of rolling your own.
Event looks like fairly low-level primitives, but also multiple
implementations. On a side note, why does everybody love talking about LIFO
queues instead of Stacks.
- The big use-case is networking. They give you “green” versions of sockets,
select(), SSL, DNS, etc.
- Good WSGI support:
gevent.wsgi wraps a C lib, doesn’t do streaming, pipelining, or
gevent.pywsgi is a little slower (does 75% of the reqs/s) but is
The “Real-Time Web”
- No page refreshes
- Server push
- Examples: Chat, analytics, etc.
- Implementations: Flash, polling, Web Sockets, and more!
- Socket.io handles papering over all the different
implementations and picking the best one based on the browser, etc.
- There’s a Gevent library to give you Socket.io support (nice!)
ZeroMQ (wonder how this fits in)
- I watched Zed’s talk; ZeroMQ is cool.
- ZeroMQ “sockets” are message-based, delivery is via a dedicated communication
- Supports TCP, inproc, unix,
- Different types of sockets: REQ/RES, PUSH/PULL, PUB/SUB, et al
pyzmq is great, but you need
- Be careful when forking or otherwise using
multiprocessing; Gevent and ZeroMQ both have
- VERY NICE! He’s using the module pattern in his sample JS. Hooray for
high-quality sample code.
- The slides for this talk are up on SlideShare.
Cython: Pythonesque C Extensions (Matt Hagy)
- You need a
- You can use this to get a huge perf win by using Cython, but
you can also use it to just make it easier to wrap existing C libraries.
- It’s almost a superset of Python (I wonder what prevents it from being
a proper superset).
- Special features for working with NumPy arrays.
- Neat: a language-level feature for thread-level parallelism. Since it’s C,
you don’t have to hold the GIL.
Writing Acceptance Tests with Freshen (Doug Hellmann)
- Features - they group Scenarios
Given to declare preconditions
Then to declare expected outcomes
- Supporting Code - applies the test definitions to the application under test
- You can use a “Background” to declare your “globals”
- Cool feature: Python’s multi-line strings
- The process of applying the tests to your system is interesting. I could
explore this for quite a while, I think. Maybe I should show it to the
QA guy at work.
- The actual tests run inside of
nose as a plugin. It detects
.feature files as “Features” and processes them accordingly.
- Parameterized testing are handled via “Scenario Outlines” (I hope it works
well, because parameterized tests are a total pain in JUnit).
- It is very new (only on v0.2), so you probably want to run with the version
From the Q&A
- The error-handling, etc., seems to be much better than Lettuce, hence going
with it instead of Lettuce.
Brandon Writes a Program… LIVE!
- PyEphem is a module that Brandon maintains and sounds cool
for astronomy buffs.
- Astronomy is fun.