The new Python asyncio module aka “tulip”¶
Asynchronous becomes very popular nowadays. The Javascript language has node.js, Erlang language has XXX, the Go language has Goroutines and Channels. What about Python?
asyncio projects¶
- Python 3.4: asyncio documentation
- Python 3.3: Tulip project homepage
- Python 2.x: Trollius
asyncio event loops¶
- ZeroMQ
- geventreactor
- gevent3
- greenlet https://github.com/1st1/greentulip
- libuv: rose, a PEP-3156 compatible event loop
- Qt:
- Mark Harviston’s PEP 3156 Event-Loop with Qt: https://github.com/harvimt/quamash
- Twisted qt4reactor: https://github.com/ghtdak/qtreactor/blob/master/qt4reactor.py
- Twisted qt4reactor: http://bazaar.launchpad.net/~qt4reactor-dev/qt4reactor/trunk/view/head:/qt4reactor.py#L55
- Tk:
- See Dino Viehland ‘s talk at Pycon US 2013
- https://us.pycon.org/2013/schedule/presentation/62/
- http://www.youtube.com/watch?v=oJQdX_w1vXY
- wxPython: http://twistedmatrix.com/documents/12.3.0/core/howto/choosing-reactor.html#auto13 and http://wiki.wxpython.org/wxPythonAndTwisted
- Tornado: experimental asyncio support built right into it.
- Glib: gbulb; include Gtk+ and GApplication event loops (suitable for GTK+ applications)
Talks about asyncio¶
- “Tulip: Async I/O for Python 3” by Guido van Rossum, at LinkedIn, Mountain
View, Jan 23, 2014
- Record of a Google Hangout (2h22)
- Slides
- “Tulip: Async I/O for Python 3” by Guido van Rossum, Oct 29, 2013 at Twitter University for the San Francisco Python User Group
- Tulip or not Tulip by Jose Ignacio Galarza, Pycon Spain 2013, Nov 26, 2013. Nice introduction to Tulip.
- PEP-3156: Async I/O en Python (spanish), by Saúl Ibarra Corretgé, , Pycon Spain 2013, Nov 2013.
- “PyCon 2013 Keynote” by Guido van Rossum, Mar 20, 2013
- Youtube video
- LWN report: PyCon: Asynchronous I/O
- “Python Async IO Horizon” by Lukasz Dobrzanski, Jan 17, 2014
- “A deep dive into PEP-3156 and the new asyncio module” by by Saúl Ibarra Corretgé, Feb 03, 2014 (FOSDEM, Bruxelles)
Tulip projects¶
Low-level libraries¶
What?
- sockets (network), stream (TCP) and datagram (UDP)
- pipes (subprocesses)
- files
- signals
Operating system synchronous I/O multiplexing, I/O event notification facility:
- select
- poll
- epoll: Linux
- kqueue: FreeBSD, Mac OS X, NetBSD, OpenBSD, DragonflyBSD
- devpoll: Solaris
- Windows proactor (IOCP)
C libraries:
- libuv
- libev
- libevent, libevent2
Python libraries:
- pyuv (libuv)
- pyev (libev)
- pyevent (libevent)
Common features:
- asynchronous DNS resolution
- multiplexing
Coroutines¶
Conference:
- PyCon 2011: An outsider’s look at co-routines by Peter Portante, Pycon US 2011:
Projects:
- Toro: Tornado coroutines
- The difference between yield and yield-from
- gevent: coroutine-based Python networking library
- greenlet: spin-off of Stackless, a version of CPython that supports micro-threads called “tasklets”
- fibers: lightweight concurrent multitasking
High-level libraries¶
- gunicorn (sync, eventlet, gevent, tornado): Gunicorn ‘Green Unicorn’ is a Python WSGI HTTP Server for UNIX
- diesel
- uwsgi
- concurrence
- Tornado: web framework and asynchronous networking library, “ideal for long polling, WebSockets”
- Twisted: event-driven networking engine
- eventlet
- ZeroMQ
- gruvi (documentation): Synchronous evented IO with pyuv and fibers, based on the PEP 3153: Transport-protocol
- App Engine NBD by Guido van Rossum
- obelus by Antoine Pitrou: Protocol implementation of the Asterisk Manager Interface and Asterisk Gateway Interface
Python builtin modules¶
- multiprocessing: Python 2.6
- concurrent.futures: Python 3.2 (Pool of threads/processes)
Concurrency¶
- Unyielding by Glyph, a Twisted developer, February 2014
- The Secret to 10 Million Concurrent Connections -The Kernel is the Problem, Not the Solution
Issues with eventlet¶
- SQLAchemy: MySQLdb + eventlet = sad
- OpenStack reaction when adding sleep(0) fixes an eventlet test
- Read “What’s wrong with eventlet?” section of Use the new asyncio module and Trollius in OpenStack