maven: good ideas gone wrong

I’ve had plenty of time to reflect on the state of server side Java, technology, and life in general this week. The reason for all this extra ‘quality’ time was because I was stuck in an endless loop waiting for maven to do its thing, me observing it failed in subtly different ways, tweaking some more, and hitting arrow up+enter (repeat last command) and fiddling my thumbs for two more minutes. This is about as tedious as it sounds. Never mind the actual problem, I fixed it eventually. But the key thing to remember here is that I lost a week of my life on stupid book keeping.

On to my observations:

  • I have more xml in my maven pom files than I ever had with my ant build.xml files four years ago, including running unit tests, static code checkers, packaging jars & installers, etc. While maven does a lot of things when I don’t need them to happen, it seems to have an uncanny ability to not do what I want when I need it to or to first do things that are arguably redundant and time consuming.
  • Maven documentation is fragmented over wikis, javadoc of diverse plugins, forum posts, etc. Google can barely make sense of it. Neither can I. Seriously, I don’t care about your particular ideology regarding elegance: just tell me how the fuck I set parameter foo on plugin bar and what its god damn default is and what other parameters I might be interested in exist.
  • For something that is supposed to save me time, I sure as hell am wasting a shit load of time on making it do what I want and watching it do what it does (or not), and fixing the way it works. I had no idea compiling & packaging less than 30 .java files could be so slow.
  • Few people around me dare to touch pom files. It’s like magic and I hate magicians myself. When it doesn’t work they look at me to fix it. I’ve been there before and it was called ant. Maven just moved the problem and didn’t solve a single problem I had five years ago while doing the exact same shit with ant. Nor did it make it any easier.
  • Maven compiling, testing, packaging and deploying defeats the purpose of having incremental compilation and dynamic class (re)loading. It’s just insane how all this application server deployment shit throws you back right to the nineteen seventies. Edit, compile, test, integration-test, package, deploy, restart server, debug. Technically it is possible to do just edit, debug. Maven is part of the problem here, not of the solution. It actually insists on this order of things (euphemistically referred to as a life cycle) and makes you jump through hoops to get your work done in something resembling real time.
  • 9 out of 10 times when maven enters the unit + integration-test phase, I did not actually modify any code. Technically, that’s just a waste of time (which my employer gets to pay for). Maven is not capable of remembering the history of what you did and what has changed since the last time you ran it so like any bureaucrat it basically does maximum damage to compensate for its ignorance.
  • Life used to be simple with a source dir, an editor, a directory of jars and an incremental compiler. Back in 1997, java recompiles took me under 2 seconds on a 486, windows NT 3.51 machine with ‘only’ 32 MB, ultra edit, an IBM incremental java compiler, and a handful of 5 line batch files. Things have gotten slower, more tedious, and definitely not faster since then. It’s not like I have much more lines of code around these days. Sure, I have plenty of dependencies. But those are run-time resolved, just like in 1997, and are a non issue at compile time. However, I can’t just run my code but I have to first download the world, wrap things up in a jar or war, copy it to some other location, launch some application server, etc. before I am in a position to even see if I need to switch back to my editor to fix some minor detail.
  • Your deployment environment requires you to understand the ins and outs of how where stuff needs to be deployed, what directory structures need to be there, etc. Basically if you don’t understand this, writing pom files is going to be hard. If you do understand all this, pom files won’t save you much time and will be tedious instead. You’d be able to write your own bash scripts, .bat files or ant files to achieve the same goals. Really, there’s only so many ways you can zip a directory into a .jar or .war file and copy them over from A to B.
  • Maven is brittle as hell. Few people on your project will understand how to modify a pom file. So they do what they always do, which is copy paste bits and pieces that are known to more or less do what is needed elsewhere. The result is maven hell. You’ll be stuck with no longer needed dependencies, plugins that nobody has a clue about, redundant profiles for phases that are never executed, half broken profiles for stuff that is actually needed, random test failures. It’s ugly. It took me a week to sort out the stinking mess in the project I joined a month ago. I still don’t like how it works. Pom.xml is the new build.xml, nobody gives a shit about what is inside these files and people will happily copy paste fragments until things work well enough for them to move on with their lives. Change one tiny fragment and all hell will break loose because it kicks the shit out of all those wrong assumptions embedded in the pom file.

Enough whining, now on to the solutions.

  • Dependency management is a good idea. However, your build file is the wrong place to express those. OSGI gets this somewhat right, except it still externalizes dependency configuration from the language. Obviously, the solution is to integrate the component model into the language: using something must somehow imply depending on something. Possibly, the specific version of what you depend on is something that you might centrally configure but beyond that: automate the shit out of it, please. Any given component or class should be self describing. Build tools should be able to figure out the dependencies without us writing them down. How hard can it be? That means some none existing language to supersede the existing ones needs to come in existence. No language I know of gets this right.
  • Compilation and packaging are outdated ideas. Basically, the application server is the run-time of your code. Why doesn’t it just take your source code, derive its dependencies and runs it? Every step in between editing and running your code is a potential to introduce mistakes & bugs. Shortening the distance between editor and run-time is good. Compilation is just an optimization. Sure, it’s probably a good idea for the server to cache the results somewhere. But don’t bother us with having to spoon feed it stupid binaries in some weird zip file format. One of the reasons scripting languages are so popular is because it reduces the above mentioned cycle to edit, F5, debug. There’s no technical reason whatsoever why this would not be possible with statically compiled languages like java. Ideally, I would just tell the application server the url of the source repository, give it the necessary credentials and I would just be alt tabbing between my browser and my editor. Everything in between that is stupid work that needs to be automated away.
  • The file system hasn’t evolved since the nineteen seventies. At the intellectual level, you modify a class or lambda function or whatever and that changes some behavior in your program, which you then verify. That’s the conceptual level. In practice you have to worry about how code gets translated into binary (or asciii) blobs on the file system, how to transfer those blobs to some repository (svn, git, whatever), then how to transfer them from wherever they are to wherever they need to be, and how they get picked up by your run-time environment. Eh, that’s just stupid book keeping, can I please have some more modern content management here (version management, rollback, auditing, etc.)? Visual age actually got this (partially) right before it mutated into eclipse: software projects are just databases. There’s no need for software to exist as text files other than nineteen seventies based tool chains.
  • Automated unit, integration and system testing are good ideas. However, squeezing them in between your run-time and your editor is just counter productive. Deploy first, test afterwards, automate & optimize everything in between to take the absolute minimum of time. Inserting automated tests between editing and manual testing is a particularly bad idea. Essentially, it just adds time to your edit debug cycle.
  • XML files are just a fucking tree structures serialized in a particularly tedious way. Pom files are basically arbitrary, schema less xml tree-structures. It’s fine for machine readable data but editing it manually is just a bad idea. The less xml in my projects, the happier I get. The less I need to worry about transforming tree structures into object trees, the happier I get. In short, lets get rid of this shit. Basically the contents of my pom files is everything my programming language could not express. So we need more expressive programming languages, not entirely new ones to complement the existing ones. XML dialects are just programming languages without all of the conveniences of a proper IDE (debuggers, code completion, validation, testing, etc.).

Ultimately, maven is just a stop gap. And not even particularly good at what it does.

update 27 October 2009

Somebody produced a great study on how much time is spent on incremental builds with various build tools. This stuff backs my key argument up really well. The most startling out come:

Java developers spend 1.5 to 6.5 work weeks a year (with an average of 3.8 work weeks, or 152 hours, annually) waiting for builds, unless they are using Eclipse with compile-on-save.

I suspect that where I work, we’re close to 6.5 weeks. Oh yeah, they single out maven as the slowest option here:

It is clear from this chart that Ant and Maven take significantly more time than IDE builds. Both take about 8 minutes an hour, which corresponds to 13% of total development time. There seems to be little difference between the two, perhaps because the projects where you have to use Ant or Maven for incremental builds are large and complex.

So anyone who still doesn’t get what I’m talking about here, build tools like maven are serious time wasters. There exist tools out there that reduce this time to close to 0. I repeat, Pyhton Django = edit, F5, edit F5. No build/restart time whatsoever.

33 Replies to “maven: good ideas gone wrong”

  1. Maven is a bit harder than Ant to grasp at the first glance, but if you already have some friends/co-workers who are familiar with Maven it is more time-saving than Ant 🙂
    You may also take a look at Apache Buildr, as an alternative to Maven.

    btw,
    1/ you can skip Maven unit tests with
    -Dmaven.test.skip=true

    2/ Maven supports incremental builds, IIRC.

    3/ About hot deploy or runtime editing stuff, that is a feature of IDE and/or the web server, otherwise Maven will be bloated 😉 . i.e. Maven is not supposed to be blame for this

    1. I know all this but it isn’t really helping that much. The point is that it is slow despite having a (very necessary) means to skip tests, run offline, and do incremental builds. Additionally, you have to jump through hoops to make it do the right thing here. On average, every time I execute a maven command, I lose at least a minute.

      Regarding maven feature set, application server feature set and IDE feature set. That’s precisely the point. The problem is that these are three different worlds and you will spend a lot of time watching these worlds catch up with each other. In a perfect world, there would be no need for all this pointless moving around of files, triple bookkeeping for classpaths, etc.

  2. I’m currently trying to get Eclipse – Maven – GWT running, an out of sheer frustration I googled “maven hell”, landing here. You’re speaking my mind totally – I spent hours trying to solve simple things like project build path setups and classpath settings. Maven’s “I believe I need to do a full workspace build now” in combination with GWT basically halts my machine for 5 minutes on 100% IO every time it happens.

    Maven eclipse integration is a major nightmare, we came to the point where you couldn’t hot deploy anymore to an application server since maven and wtp didn’t like to play together. In my experience, using maven with eclipse is a massive productivity killer.

  3. I’m at the point where ‘fuck maven’ is the deciding opinion. Seriously, get the job done and then worry about maven’s feelings. There’s neither wisdom or comfort in doing things the maven way.

    Ignoring maven has been a liberating driver for productivity. I’m goal oriented, just compile this shit the fastest way possible and get out of my way.

  4. i came across your post by googling “why the fuck do i need maven” while patiently waiting for my 15-minute build to finish timeouting. thank you for expressing everything i feel.

  5. i came across your post by googling “why the fuck do i need maven” while patiently waiting for my 15-minute build to finish its timeouts. thank you for expressing everything i feel.

  6. FUCK MAVEN. IT’S A DAMNED VIRUS! I HATE MAVEN PROPAGANDA PEDDLERS, THEY CAN ALL STICK IT!

    AND ALSO, FUCK ALL OSS PROJECTS which force you to build and install the MAVEN VIRUS in order to build their code. Not gonna participate in those loser projects, sorry, I have better things to do. Provide an ant build which “just works” without major hassles, or no free code contributions.

  7. Maybe you have a point when talking about relatively simple projects. When your unit tests, continuous build tools, standardisation, etc work for your complicated process you start to wonder where Maven has been all your life.
    Maven made my developing life a lot easier. Every tool sucks if you use it wrongly.

    1. The point is that our project is apparently too complex for maven. It actually becomes major overhead rather than a time saver.

      It’s kind of funny how many people keep pointing out that I’m the problem rather than maven. It must be your project or it must be you. I say no, it’s maven that is the problem. When I read blog posts promising that “maven 3 speeds up builds by at least 40%” my conclusion is that it was indeed a major waste of time and that I was right all along.

    2. “Every tool sucks if you use it wrongly” – not agree with this sentence.
      The best tools should be designed in a way that it will be very difficult to use them wrongly. The basic things I put in my goals of writing a new software/tool is: how to prevent the user from using it wrongly.

  8. windows install doc for Maven talks about a directory structure ( \bin etc ) which bears no resemblance to what you download. IT IS COMPLETE BOLLOX – I can download Netbeans, install and I’m DEPLOYING apps in minutes. This fuckwit console mentality that we should waste weeks searching for obscure posts on fixing the obligatory issues with free tools….. some fucking adolescent hacker kudos bollox…..fuckit ! if you use Maven your software will eventually fizzle into fuckwit land.

    1.  I typed in ‘maven is shitty’. You’re on the second place in the ranking, the first is some mvn plugin named ‘shitty’ :D.

      Thanks a lot for that blog, which I can read while putting togather my mobicents project :D.

  9. I tried to tell the clowns where I work that maven is breaking Eclipse’s incremental compiler, but no one seemed to care.
    The kinds of people who really, really hate maven are the power users.  These are incidentally the same people who have to debug the system when maven screws it up (not the incompetent fanboys who were pushing the crap in the first place).

    1.  The maven (and JPA – but that is another blog matter) fans are all slaves to marketing.  A project comprehension tool? How does a directory structure and a huge xml file make me comprehend a project?  All maven ever did to me was to make me confused.  Why is the build failing when it built OK the last time (and I haven’t changed anything)?  And why does it take 15 minutes to run (ant takes 3.5).  Use maven at your own peril.

  10. Actually some of the ideas you are describing about the application server loading code directly is maybe the strongest point for using the play framework(google it).

    It is good that after all we are not the only people that hate building code externally. I had some fun reading your post, thanks for that 🙂

  11. Yeah, I’ve been dealing with maven all day.  I kept thinking to myself that this was vastly over-complicated and a huge waste of time, but I was trying to entertain new technologies.  I couldn’t agree more with your post.  Fuck Maven.   

  12. pom pom pom … i resisted to you..still using ANT, I make a complex ant file from little xml (package name + names of jars + optional extra ant xml) with xslt. My Xslt contains the model of a quite complex ant build file, that makes build, bin, src, zip, test, testzip etc.. 1 time a year perhaps i change something, but not recently. Yes, it does not download the files. But try to install JAI (other craziness) with Maven.. I liked so much RESIN because I was doing with it TextPad in app folder + F5, like the scripts languages that you mention…for classes, not for jsp 😉

  13. Thanks bro… I came here typing in google ‘maven crap’, because it looked like shit when it was mavem 1, then when it was maven 2 – I started to think the way you wrote, but not in such concise way, and agter 2 year pause I try maven 3 – 5 min tutorial, and the 1st command fails (archetyme:generate) :))) WTF?!!!

  14. I feel your pain. Five years and this post still resonates. Maven is essential in huge projects like the one on which I’m working now, but it is still as brittle as ever. The documentation is some of the worst I’ve seen, especially for such a widely adopted tool. Right now, intall is telling me it can’t find a dependency which I can plainly see in my local repo and which M2Eclipse finds without problems. It’s especially annoying that these are both modules under the same POM project. BTW, I googled “maven gone stupid”.

    1. Yep, I still use (not much choice generally) it but aggressively fight back any form of customization, excessive profile copy pasting, and weird/obscure plugins from hell on any project I join. It’s the only way to stay sane. Anything complicated, I prefer to handle with a handful of lines of script on jenkins rather than hundreds of lines of xml to pull in a gazillion half working plugins that nearly but never quite do what you need.

  15. Just finishing up an automated prep / offline maven build process. My hair is bleeding after 20h straight. I come from a land of Makefiles and had a hard time understanding why would someone need ant … but this … this … putrid, evil, vile mosaic of shitbits makes me want to murder someone, by making him read my pom.xml files without having a single sip of water. Btw can some maven zealot please explain to me why would I have to have repository configuration present in my settings.xml for offline build to run without exploding in shower of sparks? Please consider that all dependencies are correct in the local repository … how do I know, well I put bullshit credentials for the artifactory login, and ever a skeptic unplugged the network cable (no artifactory is not local) and it worked.

  16. As I said earlier (much earlier) every tool sucks if you use it wrong.
    From what I read, much frustration comes from using the m2Eclipse plugin, which is an absolute horrible tool. It also puts Mavens concept of convention over configuration completely upside down. Use a proper IDE when working with Maven 🙂

    The project I’m working on now, it contains 67 modules in 10 main modules. In one week time I created a Maven chained process in combination with Jenkins that:

    builds, tests, versions, tags, branches, deploys to any given environment except for production, checks, starts / stops servers, …

    The process also has ‘customisations’ for every module.

    This seems like quite a lot of config, however not a single pom contains more than 100 lines of xml. It’s basically a deal of inheritance and overriding, which are for programmers not hard concepts to grasp. If there are organisations where almost no one dares to change the poms then that organisation should seriously consider changing their personel setup.

    As with performance… Quality requires running your tests, properly. Running those tests performance-wise is much more dependent on the IT systems in general than the tools used. CPU, I/O, … Those tests are for every project I worked on the main factor of ‘timeloss’ in builds.

  17. Thanks for confirming my suspicions. Laughing at the “google search” posts.
    My very first maven build followed by the following google search: “is maven a bloated piece of shit?” landed me here.

  18. Same thing is happening in JS land, everyone and their mother wants to invent their own build system.

    Stay simple folks, avoid layers of abstraction as these are just where things can go wrong, leverage the lowest common denominator: bash/bat scripts.

    Avoid dependencies, if something needs to be downloaded it means that’s just a point of failure and at the worst possible time of the project. Also lots of time newer versions of packages are downloaded as part of the build which can and will introduce bugs that didn’t exist.

    So try this for a change, avoid the build and dependency requirement completely, cloning the repo is the only step an end developer should be required to do. All the dependencies, the minified files, etc. are tracked as part of the repo; and in my experience this works rather well.

    On a bright side, problems with build abstraction are not JAVA specific, these are issues in any language: http://danielsokolowski.blogspot.ca/2012/08/setuptools-includepackagedata-option.html

Leave a Reply