Clojure on Android

Clojure Gazette

Issue 1.9 - May 10, 2012

editorial

Clojure on Android

I am very pleased with this week's issue!Daniel Solano Gómez andAlexander Yakushev graciously agreed to do interviews.

Daniel SolanoGómez worked magic and brought dynamic compilation to Android. You can test it for yourself with the Clojure REPL app in the Android storeand he presents a lot of the challenges in his talk at Clojure/Conj 2011.

Alexander Yakushev was accepted to work on the Clojure on Android project in Google Summer of Code. You can read the proposal here.

Now, on with the interviews!

Eric Normand

Daniel Solano Gómez

Clojure Gazette:
Hi Daniel,

Here is the first question:

Could you tell us a little about yourself and how you got into Android development with Clojure?

Daniel:
I have enjoyed programming for about as long as I can remember, and I have now been doing it professionally for nearly fifteen years. In my career, I have done primarily object-oriented software development using Java and C++.

Unwittingly, I had started doing some functional-style programming in C++, but it wasn't until two and a half years ago that I came across Clojure. Clojure was my first exposure to Lisp and to really understanding functional programming. As someone who never studied computer science in depth, it has been a mind-opening experience.

In January of 2011, I decided that I should learn about mobile development, as it seemed to be a growing market. The main two choices were Android and iOS. Given that I didn't have any Macs and I was already familiar with Java, I decided to go with Android.

It wasn't long before I decided that I wanted to be able to use Clojure for Android programming. I think some people had already tried it before me, but I decided to make the Clojure REPL for Android, and that was my first real Clojure and Android project.

Clojure Gazette:
You overcame a huge hurdle by getting the Dalvik VM to dynamically compile Clojure and load the bytecode. This allows one to, for instance, run a REPL natively on Android. Was this step necessary? For instance, could the compilation be done on another machine and pushed to the device, similar to how Himera compiles ClojureScript on a server? How would that affect the development workflow?

Daniel:
Is it necessary? No, I suppose not. Before I got the dynamic compilation working, it was always possible to AOT-compile and process the code and then deploy it to a device. However, that's not the dynamic development experience Clojure users have grown to expect.

Although I am not very familiar how Himera works in detail, I think it could certainly work. However, I think the main challenge of this approach is that it would probably be more complex, both to develop it and to use it. In addition to the application running on a device or emulator, you would need some sort of compile server running on your workstation. By being able to include the JVM to Dalvik bytecode component as part of the application, it is possible to work with it as if it were any other Clojure application. So long as your editor supports something like nREPL or VimClojure's Nailgun protocol, it just works.

The one exception to this may be if you are doing ClojureScript development with Android. I don't have much experience with this, but I have heard that it runs using the Scripting Layer for Android (SL4A), and maybe even with PhoneGap. So, if someone is interested in creating more of a mobile web application, ClojureScript is certainly an option. Himera may be able to fit in better with someone taking this approach.

Clojure Gazette:
Very interesting and complete answer. I have played with the REPL app a bit on my phone and it is indeed the Clojure I know and love.

I am curious to understand this more deeply. The way I would naively imagine wanting to develop on a small device would be for me to sit down at a desktop, type some code, have it compile on the desktop which then pushes the code to the small device sitting next to my keyboard. Basically, I can drive the device from my desktop. Wouldn't that play to the advantages and weaknesses of both types of devices?

Daniel:
Well, there is certainly an argument to be made for it. Compilation on a computer is faster, and there are some macros that overflow the device's stack during compilation (the 'for' macro is particularly notorious in the regard). Unfortunately, even if the maximum amount of work is offloaded to a workstation, there is still a fair amount left to do on the device.

In particular, one technical limitation of the Android platform's API is that in order to load new code during runtime, it must be:

1. Saved to disk.
2. The saved code must go through an optimisation process that presumably loads the bytecode from disk, manipulates it, and then saves the optimised bytecode to a new file.
3. Finally, this optimised bytecode can be loaded into memory, and a new class may be loaded.

I have not measured to see how much overhead this adds in comparison to the rest of the compilation process, but it's going to continue to be a performance issue until Dalvik has first-class support for dynamic bytecode loading.

All that being said, the main reason I didn't do it that way is that it never occurred to me. I simply saw an easy way hook a JVM to Dalvik VM component in one of Clojure's classes.

It may be an interesting project to see what Himera could do for native Android development, but it's not something that I am going to pursue. There are a couple of reasons for this:

1. Given the overhead of dynamic code loading, I see dynamic compilation primarily as a development feature. Taking a few seconds to load something is probably too slow for any user-interactive application, but it's at least an order of magnitude faster than the typical Android compile and redeploy cycle.

2. These devices aren't so small any more. The latest phones have four cores running well over 1 GHz and a gigabyte of RAM. You can also run the latest Android emulator using hardware virtualisation. A lot has changed, and things will only get bigger and faster with time.

Clojure Gazette:
I see. I think compiling on the device is awesome, by the way. Especially as devices become more capable. It will pay off in the future and give Android developers another great tool.

On another subject: you are working on a book about non-Java development for Android. How is that coming? Are there things that Clojure could learn from other languages? And vice-versa?

Daniel:
It's coming along fine. I had originally started with a general Android book that uses alternative JVM languages, but we decided to change focus to native development using C and C++. My plan is to finish writing it by the end of the summer. Afterwards, I may still do some e-books that concentrate on Anroid with Clojure and Scala.

I think one of the big things in which Clojure can improve is in its tooling support. A little over a year ago, as I was preparing for a talk at Texas Linux Fest, I tried developing the same application using different approaches, including three JVM languages: Clojure, Scala, and Mirah. The one thing that really surprised me is how easy it was to develop with the Android plug-in for Scala's simple-build-tool (SBT). I would consider it even better than the standard Java toolchain of the time. As a result, I am really excited about Alexander Yakushev's Google Summer of Code project. I hope that his work will make it much easier to develop for Android using Clojure.

Going the other way around, I think that languages such as Clojure and Scala can help bring new perspectives to Android development. Android's APIs are naturally very Java-centric. As such, as the platform grows richer with features, I think it begins to suffer from the limitations of Java, such as a lack of lambdas.

For a Clojure developer, especially one with a lot of previous Java experience, it's way too easy to slap a few lambdas over the Android API to make it more pleasant. However, as the underlying platform grows in complexity, these sorts of measures become less productive. In particular, my experience with development with Android is that it is increasingly asynchronous and event-driven. Something I have been mulling over the last few weeks is making application development much more declarative and explicitly event-driven. For example, instead of overriding an 'onPause' method or providing a listener on a widget, you could use core.match against the different types of events in the application.

Another idea I have toyed with is some sort of DSL for Android GUIs, analogous to what Hiccup provides for HTML. Android uses a declarative, XML-based approach that has a lot of benefits, but I think that is possible to improve on that significantly using Clojure. Android's native XMl approach suffers from two drawbacks:

1. With complex projects, the interface for a single screen might be duplicated in several different XML files to handle different device configurations. While there are things you can do to factor out common parts, it can easily become a bit of a mess to manage.

2. Since Android applications typically inflate their UI from these XML files, it means you cannot take advantages of things like dynamic recompilation to tweak your UI. Any change to the layout requires a full redeploy cycle.

With layouts declared in Clojure data structures, you can resolve both of these issues. You could easily compose pieces of layout or mix logic in with your structure so that all of the different variations of a single layout can be defined in one place. Secondly, you could tweak this at runtime to interactively get your UI set up the way you want it. Once finalised, you could then generate XML from the same source for the release of the application.

Clojure Gazette:
That sounds really powerful. I imagine that dynamic layouts would make Clojure unique among Android languages. Are the Android core developers receptive to this kind of tinkering?

Daniel:
I don't know. I imagine there a lot of smart people at Google who may be receptive to these ideas, but I doubt that they will become the norm any time soon. It will be up to communities like those built around Clojure and Scala to experiment with, prove, and promote these techniques.

Clojure Gazette:
It was really nice interviewing you. I learned a lot.

Daniel:

Thank you, it has been a pleasure. It's always fun to talk about Clojure.

Clojure Gazette:
Is there anything else you would like to mention that we did not talk about?

Daniel:

One thing that I had forgotten about when talking about event-driven and dynamic user interfaces, is that it is worth looking into what can be learned from Dave Ray's Seesaw. He has done a great job of going beyond simply wrapping Java.

Also, I would like to thank the greater Clojure community for challenging many of my conceptions on to how to create software, and inspiring me to become a better programmer.

Clojure Gazette:
If someone would like to follow the progress or help out, where should they go?

Daniel:

There is a mailing list for Clojure on Android, which is probably the best place for discussions, announcements, and questions. I periodically post there when I get the chance to fix an issue or add a new feature. If anyone publishes an Android application using Clojure, I encourage them to post about it there.

Finally, I'd like to point out that Clojure on Android is still a nascent effort. Even small contributions can help propel Clojure to being a first-class choice for Android development.

Clojure Gazette:
Thanks, Daniel! This was a great interview. I think people will like it.

Daniel:
Your welcome, Eric. I enjoyed the interview, and I hope others will, as well. Thank you for the opportunity to talk about Clojure and Android.

Alexander Yakushev

Clojure Gazette:
Hi Alex,

First question!

You recently were accepted to participate in Google Summer of Code to create tools for Clojure development on Android. Please tell us a little about yourself, including how you started with Clojure and what interests you about the GSoC project you will be working on.

Alexander:
I am from the category of people who started programming in early childhood. At school I took part in different ACM-like programming contests. You know, the ones that favor the execution speed above all other factors. Of course Pascal and C were the only options there. When I remember those times and the code I was writing it sends shivers down my spine. That was not programming at all - though we were told it were

• it was about doing something the fastest and dirtiest way possible. I guess such experience leads to this mindset in the community about the inferiority of "slow" languages.

Anyway, it was my third year in university when I first encountered Clojure. That was two years ago. I've been already interested in Lisps and functional programming at that time and was learning the bits of wisdom from SICP and PCL. After all years of Pascal, C and Java it was wielding the immense amount of power. But there were a few things that bugged me about this power. Common Lisp's "reader macros all the way" approach made any harder-than-primitive code look like APL. Lack of namespaces led to long symbol names. Scheme on the other hand was lean and mean but required too much to implement from scratch in order to use in real world. Considering all this Clojure was a gift from above for me.

My primary interest in this GSoC project is basically because I develop for Android as well and I know the pain of constantly recompiling the project after each minor change. After the experience with REPL in Lisps it is just too much to bear. So that's why I was incredibly excited when I saw the video of Daniel's talk at Clojure/Conj 2011. I was twice as excited when Daniel signed up as a mentor for this summer's GSoC. Three times excited I felt when I was accepted for the project. So I may only guess how many times of excitement I might get if this project ends up successful. :-)

Clojure Gazette:
You mention the pain of recompiling and (I assume) redeploying to your device in order to test a small change. The GSoC proposal seems to attack three pain points: setting up a project, dynamic development, and deploying. What is your vision for the Clojure + Android development cycle?

Alexander:
Ideally it should be as easy as with Clojure for JVM. Practically there are a couple of difficulties, and tackling them is one of the tasks for my GSoC project.

Here's an example how I see it. You do `lein droid new` from your terminal (Eclipse plugin, whatever). A default project is created where you only have to specify the path to the Android SDK and the initial libraries you want to use. Then you do something like `lein droid doall` and your project gets compiled, built, deployed and run on the Android device. After that you fire up the nREPL client of choice (Eclipse CCW, Reply, I hope that Emacs will eventually get nREPL extension too) and here it is - your dynamic development workset. You can write and compile functions, change UI and its behaviour on the fly

• this is a fully functional REPL with a few exceptions.

For instance, we don't yet have a viable idea of how to add Activities dynamically. Dynamically changing the UI by editing XML layouts is also a no-go - that is why one of the points in my proposal covers the creation of an idiomatic UI toolkit (like Seesaw for Java Swing). The same is also true for strings.xml and i18n but these things could be substituted by customly built ones without much pain. Yet things that are strictly tied to the AndroidManifest are harder beasts to tame.

Such changes (defining a new Activity, or adding a new dependency library to the project) would require recompilation and redeployment. Since the compilation/building time of the Android/Clojure project takes

about 1-2 minutes, doing this often will certainly hurt the productivity. That's why yet another goal of my summer project is to reduce to minimum the cases that require recompilation.

Clojure Gazette:
Wow. That sounds like it would be very easy. Are there any other languages out there that can do dynamic development on Android?

Alexander:
AFAIK, Clojure is currently the only language that can be dynamically compiled on Android platform. Of course you can use some interpreted languages like Lua and Ruby but they lose at performance greatly. Clojure's performance on Android is on par with Java, having some troubles with the start-up time and heap/stack usage, but overall quite competitive.

I think the ability dynamically develop native apps with Clojure is the most important. Replacing Java as the language of your Android development means you will have trade offs. Java and its esosystem around Android is really advanced, with all the Lints, graphical UI previewers, debuggers and profilers. The question is what you get instead. With Clojure you receive not only the fancy syntax or "first-class functions, at last!" but the whole new development paradigm.

Clojure Gazette:
Could you talk more about the challenges around creating new Activities? I am not familiar with the details of Android development.

Alexander:
Briefly speaking, an Activity is something similar to a Form/Frame on the desktop. The difference between them is that Activities replace each other on the screen by passing messages (these messages are called Intents) rather than invoking some method of the target activity. That being said all the Activities and the Intents they are triggered by should be described in a special AndroidManifest.xml file before the application is built. During the installation Android reads this file and registers Intents for respective Activities. At the moment there's really not much to do to overcome this limitation.

Clojure Gazette:
Is there anything else you would like to mention that we d id not touch on that might interest the readers?

Alexander:
Can't think of anything right now. Since most of the ideas are still just ideas there's really not much else to say. The actual deeds and the result of this project will show more than words.

Clojure Gazette:
If someone would like to follow your progress where can they do that?

Alexander:
I've set up a blog dedicated to this project: clojure-android.blogspot.com. The information I post there is not very refined so it might be boring to read through my ramblings :-). But when things start acquiring some shape I think the format of the blog will change to be more project-oriented and less "diary".

Clojure Gazette:
Thank you for doing the interview.

Alexander:

It was my pleasure! I appreciate your interest in the subject and hope that readers will find this interesting as well.