Want this course?
Leiningen and Dependencies
Course: JVM Fundamentals for Clojure
Leiningen hooks into the Maven dependency management to provide dependencies for Clojure. We take a look at how Leiningen does that. We also see how to 1) find conflicts in the dependencies; and 2) find old dependencies that can be upgraded.
So let's talk about how Leiningen manages dependencies. This is the project file that Leiningen requires this is actually one for an older project. It's got kind of like a weird structure that I've set up for it. It still works and I just wanted to show you some of the things in it.
So the dependencies in Leiningen go in the dependencies key, usually in a vector, although that's not required but that's really what people use. The dependencies are each a vector themselves. That vector includes what I usually call the Leiningen dependencies string. This is as an example this is the Leiningen dependencies string, this is obviously the version ID, it's a string.
This is the artifact ID and this is the group ID. So what Leiningen has done is make a very concise format for expressing those three things. Group ID is the namespace of the symbol, then the name of the symbol is the artifact ID and then you have a string as the version ID.
Now there are some directives you can put in, like this one has an exclusion. So what this is saying is that I want to include this library, but and then this version. But it's going to have a dependency on Guava, but I want you to exclude that because I'm going to include my own version of Guava. So, same here, LazyMap would require Clojure But I want to use a different version of Clojure so don't download the jar that LazyMap is pointing to, I want to use my own version of Clojure.
These exclusions are kind of hard to know about because when you include a library, most of the time you've got this huge tree of dependencies that you rely on. You've got the direct dependencies but also the transitive dependencies. The things that those dependencies depend on, and so on. You don't really know everything that you have to exclude but Leiningen has a feature that helps you out there. So I'm gonna switch to the Terminal, here I am in its directory.
We can see, oh there's a lot of stuff in there. You can see the project file there, same project file. So I'm gonna do lein deps :tree Now, lein deps :tree is a command, lein deps used to be important because it would download all dependencies. Now in Leiningen 2.0 all the dependencies get downloaded automatically, you don't have to do a separate command. But there is still this :tree option which will print out a tree of all the dependencies. So just as an example, I'm dependent on Ring, and I had an exclusion of jetty-util but Ring depended on Ring Core, which in turn depended on clj+time also on commons-fileupload on commons-io and you see this tree of all the dependencies that come through Ring.
We can go up here and see that Ohm depends on React that makes sense, core.async depends on tools.analyzer which depends on core.memoize, et cetera. So this is a lot of dependencies all of this has to be there for the program to run, I mean, in theory, you might never call the code that does a regex, right, you might never get there. In theory it could be run. Actually, Rick Hickey has a talk about dependencies you don't really need. You should look it up, it was at the 2016 Clojure Conj.
Okay, this is the end of it right here. This is the top of the tree. So these are all the things I've listed and these are all the transitive dependencies. But if you keep screwing up, you keep getting these other messages, sometimes. Here's where I ran the command. We see that it's telling us possibly confusing dependencies found. What it's saying is there are different versions of the jar, the same group ID, same artifact ID but different version string. That are being included in different paths on the tree.
So this one is saying, look, figwheel is depending on sidecar which is depending on pomegranate which depends on maven-aether-provider which depends on plexus-utils 2.0.6. But, in this other branch of the tree, figwheel-sidecar, pomegranate, wagon-http, wagon-http-shared4, wagon-provider-api, plexus-utils 3.0 so the 2.0.6 and the 3.0 conflict. They're just different strings, so they are telling you there could be a problem.
Because whichever one gets loaded really depends on your system settings. It depends on what order they are in class path string, it depends on how the class loader was written, so does it look in the first one, that it finds, does it collect them all and choose one in some other way because the class file doesn't have version information in it. That's all done at the pom level so the jar doesn't even know what the version is so it's actually a problem in Java that you run something in two different times and the class loader could find two different versions of the class file. It just finds one, and it says I found it.
So if there's two in there, it might be ignoring one. Because it found it in another place. This is really a source of bugs, which is why this tool exists. You need to resolve all of these issues. If you want to have a stable production environment for sure. This is saying this one, this one, this one and this one, kinda conflict and you should do something about it. So these are 3.0, 3.0, 3.0 and this one is 2.0.6 so it's not the error messages aren't super smart it's telling you you're gonna have to put an exclusion in there. Don't rely on the one that figwheel gives you.
Because it's giving you two different ones the 3.0 and 2.0.6. So just say don't even include it. So then what you need to do is make sure you are including it in a specific version at the top level explicitly So the suggestion, the best way to do it, the best practice is you add this exclusion that Leiningen is telling you.
That's really easy, I'm just gonna copy it like this. Go into Emacs, and that was on figwheel, right, and this is a plugin dependency so it's different from my production dependencies and I'm just gonna do that. :exclusions org.codehaus.plexus/plexus-utils Now that means I need to add this, and I can add this one, this 3.0 and here's the other thing, you just have to believe that it is compatible, that 2.0.6 is compatible with 3.0. I'm sorry, there is no real solution to that. You should find the one that you need to use and just stick with it. Stick with the one that you tested, that works. This one actually needs to go in the dependencies so I'll just put it here at the bottom.
Now, what should happen is if I run it again, it should tell me I should not have that particular message again. So here is the top, so now I have the different one from figwheel also. This is an old version of figwheel. And it's a plugin, so this isn't really gonna be an issue in production, this is just when I'm running figwheel on my local machine. I do want to resolve this, so we see, I'm not even going to look at this path, I'm just gonna look at the last one, this httpcore 4.1.2 and this one says httpcore 4.2. So there is a possible conflict there. Maybe a bug was fixed in the 4.2 that was still in 4.1.2 so I want the newest one. I'm gonna do what it says, I'm gonna add the exclusion Then go back to Emacs, lein-ancient, is that even in here? It's not, OK, so this is actually in my profiles there it is, OK. I'm going to put that in there httpcore or the other thing I could do is. No, I should put it in the figwheel. So it was recommending I put it in the lein-ancient one, but I'm going to put it in the figwheel. Just get it from there.
So I'm gonna put it in here so notice that this is just the symbol which contains the group ID and the artifact ID I don't need to put in the version string there. Okay, I need to copy this again. I just need the symbol, so I'm just going to copy the symbol I've already got an :exclusions, just put it like that. Now the thing is I need to add httpcore 4.2 to my dependencies. Did I get rid of that other dependencies? Yeah, I just pressed the wrong thing in Emacs. Now, scroll back up and find it this plexus-utils saved. Now when I run it, I should be able to get fewer messages let's see, come on, come on woah.
As you can see, this is an incremental process, because I found a new one now which is httpcore and commons-codec need to be excluded from clj-aws-s3. You just continuously go through this, and eventually you will get to a nice, clean, like these warnings will go away. The messages will go away, and you will just have your tree. So that's lein deps :tree. There's some more stuff that I'd like to show you. Let's say that you've got this set of dependencies and you know this software, I haven't touched it in a year at least, maybe two. So what do I need to do?
Well, we already saw that thing called lein-ancient it's a plugin. I'm going to run it and what it does is brings in your dependencies and then goes and sees if there's new versions of everything and of course it's old so there's all these new versions. I am not going to update these. I don't have the time or the energy right now to go through this old software and make sure that everything is up to date. I think if something is working, you shouldn't break it. Sometimes, you just want to know, what are the things that are old, that I could update. And that's nice, this is saying what we use and this is the new version, there you go. Look, I'm using a very old version of Clojure script, before they started doing releases. And look, 1.9.293 is available.
You know, I might consider updating that, but whenever you update software there is always problems. The thing was working, it's fine, so I'm not going to touch it right now. I think I've gone over everything, I just want to go over one more thing, this is kind if standard actually. If I go to my home directory .m2 this is where Maven gets cached. It's seen as a repository and a settings.xml this is a Maven settings file. That repository, I go into it see I've got all of the group ID's that are usually dotted the group ID's so org.clojure, so let's see if I can find the org one here. Org, there it is org, right so I go into org cd org. Now I have org.apache, org.clojure, org.postgresql org.elasticsearch, so I can go into Clojure and I can see all the Clojure projects.
This is the artifacts now, there we go. Now we go into versions of Clojure I have. Look I already have one all the way back to 1.2.0. Let's look in that one. So inside there we see the jar, we see the pom, there's sha's of both of them so you make sure you downloaded the right one without any errors, that's nice. And this is like a little cache of the Maven central repository that you have on your hard drives. So you don't have to download everything everytime. Because Maven is very good about not changing an existing jar, that you have to bump the version, that means that this cache can last forever because things aren't changing on the server and I can cache it forever, and that is great. Now the thing about the .m2 directory, if you ever do have a problem just delete it because it's just a cache, just like delete your repository directory in there. You might want to leave your settings, but the other thing is it's standard. Other software like IntelliJ the Maven plugin for IntelliJ is also using the .m2 directory.
So everybody is putting dependencies and caching them in there So that's where things go. I have often, maybe 10 times decided I'm going to delete this cache and it solves a problem, so you might want to think about doing that at some point. But don't do it just willy-nilly only do it if you think things are kinda messed up. The other thing is sometimes I want to see the jar file what's in there so we're gonna take a look at a jar file in another lesson, when we look at the anatomy of a jar file.