Reuben CummingsLambdaConf 2017 Interview
Reuben Cummings LambdaConf 2017 Interview
Reuben Cummings will be giving a talk at LambdaConf 2017. He will be speaking about Data Processing in Python.
PurelyFunctional.tv: How did you get into Functional Programming?
My foray into functional programming (FP) began shortly after picking up Python in 2011/12. I initially spent a decent chunk of time reading through the Python documentation. Upon discovering the
itertools module, I immediately saw the power of Python's FP primitives. Most people think of Python as object oriented; but it's more accurately defined as multi-paradigm.
Resources which played a pivotal role in my FP journey include the original 1986 Structure and Interpretation of Computer Programs (SICP) video lectures (and the accompanying book), Rich Hickey's "Simple Made Easy" Clojure talk, and "Turning the database inside-out" (both the blog post and talk).
Despite the fact that none of the above are Python specific, I have become a much better Python programmer by internalizing the (FP) approaches they advocate.
PF.tv: What is your talk about?
RC: My workshop, "A Functional Programming approach to data processing in Python" aims to explain how to take advantage of Python's excellent functional programming capabilities to efficiently process tabular data. Additionally, it will introduce meza: a pure Python, functional, data analysis library and alternative to Pandas. Complete with various examples, you’ll see how meza elegantly ingests and analyzes data.
PF.tv: Who is your talk for?
RC: My workshop is appropriate for people fitting a couple different profiles:
- Programmers who would like to learn how to use more FP aspects of Python
- Data scientists/analysts who would like to add new tools to their toolbox
Since this is a workshop, attendees should come with their laptop and be prepared to "get their hands dirty". I'll have a couple hands-on exercises to accompany the presentation and demos. While it isn't necessary for attendees to be FP or Python gurus, those with at least some experience in either topic will get the most out of the workshop.
And those who've never touched Python before need not fear. I'll provide links to Jupyter notebooks so that you can follow along and perform the exercises online without having to download Python or a docker image.
PF.tv: What do you hope people will take away from the talk?
RC: The pitch of my workshop is "Your days of object oriented data processing in Python are over." Developers will learn data processing techniques that are less complex, have fewer dependencies, and less resource intensive than popular existing solutions.
They will also discover how to obtain data from the web, read it into a convenient data structure, analyze and transform it, and finally write it back to disk in various formats (csv, json, xml, etc).
PF.tv: What concepts do you recommend people be familiar with to maximize their experience with the talk?
RC: It wouldn't hurt if attendees have a passing familiarity with lazy evaluation, infinite sequences, and memoization. I'll be giving Python examples which explain these concepts, but someone completely new to both Python and FP may have to pay extra attention.
PF.tv: What resources are available for people who want to study up before the talk?
PF.tv: Where can people follow you online?
PF.tv: Are there any projects you'd like people to be aware of? How can people help out?
RC: Aside from meza, check out riko. riko is a stream processing library that builds on many of the concepts introduced in meza. People can help out by giving it a whirl and letting me know what they think. The issues page currently serves as riko's roadmap. I encourage anyone who is interested to make comments, submit pull requests, or help improve the documentation.
PF.tv: Where do you see the state of functional programming in 10 years?
RC: Since I'm most familiar with Python, I'll speak on that, and specifically in regards to asynchronous I/O.
I'm really excited to see the progress that's been made with Python 3 and the asyncio module. Prior to asyncio, the primary means of writing asynchronous (concurrent) Python code were the Twisted and Tornado libraries. These libraries gave Pythonistas the ability to write synchronous looking async code (think node.js promises) without resorting to spawning multiple threads (see "Unyielding" for more on this topic).
Asyncio borrowed many concepts from Twisted and added several of its own, like async/await, that allow developers to write even cleaner and more robust asynchronous libraries.
However, due to the leaky callback API abstraction, asyncio suffers from several issues such as slow performance and poor backpressure/buffering support (see "Some thoughts on asynchronous API design in a post-async/await world" for a detailed review).
Curio, PyPy, and uvloop improve on many of these shortcomings. Yet, compared to their synchronous counterparts, async web frameworks have significantly fewer plugins. This fact makes it difficult to develop async web servers that support anything beyond the bare essential use cases.
So in 10 years, I see Python closing the gap on these fronts. By then, PyPy-STM (which introduces a Clojure-like concurrency model to Python) will be much more mature. More libraries will make use of type-hint based optimizations. And the async ecosystem will have evolved to become suitable for developing the kinds of highly concurrent systems that are written in Clojure or Scala today.
PF.tv: If functional programming were a superhero, what superpower would it have?
RC: Definitely shapeshifting (like Mystique from X-Men). The great thing about functional programming is that it can assume the functionality of virtually any programming paradigm. Concepts like memoization and recursion obviate the need for global state. There are those that even say functional programming completely removes the need for design patterns. I wouldn't go that far, but I would say FP eliminates the vast majority of "patterns" one needs to learn.
This interview is not sponsored by nor affiliated with the conference or its organizers. It is in no way official. It is simply curated and organized public information about the conference.
- Harold Carr LambdaConf 2017 Interview
- Arseniy Zhizhelev LambdaConf 2017 Interview
- # Kjetil Valle and Bendik Solheim LambdaConf 2017 Interview
- Daniel Beskin LambdaConf 2017 Interview
- John Azariah LambdaConf 2017 Interview
- Nikola Adamus LambdaConf 2017 Interview
- Fred Cecilia LambdaConf 2017 Interview
- Riccardo Terrell LambdaConf 2017 Interview
- Aldric Giacomoni LambdaConf 2017 Interview
- Gavin Whelan LambdaConf 2017 Interview