Having worked at a company with a large Clojure codebase, I just don't see Clojure growing. It's not well-suited to large projects. The dynamic-ness of it and awkward parts of the language (macros, protocols, ambiguity of laziness, etc) end up inevitably getting used in ways they aren't intended.
The idea that you need "wise programmers" is a really big problem. People leave, institutional knowledge is lost, and then the code archaeology is just more difficult than it would be where types exist as guard rails.
I think Clojure looks favorable in comparison to Java still (though perhaps less so). But with Kotlin and Scala as mature alternatives -- and Clojure without a real niche -- it's not what I would reach for.
Maybe a broader version of this point: clojure tries to be something suited both for industry use at scale and for hobbyists wanting to do freaky-deaky stuff at the same time, but those are fundamentally incompatible.
I say this as someone squarely in the latter category who has exactly the opposite problem from you: I love clojure the language and its features, but all the major libraries seem to be written by refugees from Java working at big companies who have a will to massively overengineer everything.
I'm thinking of libraries like mount and such which everyone uses for everything, and which I experience as, like, "did you want to inject your dependencies in your injected dependencies? Well first you have to do a quadruple axel inversion of control at the fifth abstraction layer out and use the fooflarb design pattern to delegate your inheritance and you thought you could even find the place in the code where it talks to the database? Good luck, sucker." Like, I always thought the point of a functional language was to not have to do any of that stuff...
Like, I vividly remember the first time I used a clojure web framework that brought in all that stuff, and I literally, I'm serious here, could not find where in the code it was actually talking to the database. Just indirection on top of indirection on top of indirection. And when the application I was building on top of that, which was supposed to be quick and dirty, started mysteriously losing data, the only thing I could do was just tear it down and rewrite with Python and flask.
I'm surprised to hear you had such a bad experience with mount--it's always struck me as one of the greatest exemplars of what Clojure allows you to do that would be impossible in Java. Managing fundamentally stateful components was always a nightmare for me in Java and Java-related languages: you'd never know which class was in charge of initializing what, would always be tearing your hair out trying to keep track of when you can assume certain components have been successfully initialized, etc.
With mount on the other hand, you tell it how to start and stop the component (and stores a reference to the component in the same var where you specified this), keeps a reference to the component in exactly one place, and then it automatically makes some reasonable assumptions about the order in which they need to be initialized, while allowing you to opt out as needed. Re: your example of finding a reference to the db, shouldn't it just have been in the var set up by a call to `defstate`? (Also, which web framework was it? Luminus?)
It's not fair to point to specific overengineered libraries and say this is the language idiom. Clojure is a small, stable language (few changes over the years) that encourages creation of DSLs through macros. It provides solid ground on which to build the abstraction you want, so it's possible to build these EE-ish indirection nightmares if that's what you need.
Clojure's design allows it to serve the needs of both hobbyists and enterprise, but the latter requires strong, disciplined developers who understand to avoid loose, hobbyist code that the language allows, and who will agree on which patterns to use in the codebase (something the hobbyist gets for free mostly). It's a punishing language for a project with high developer turnover.
If anything, I'd say that Clojure discourages the creation of DSLs through macros compared to most other Lisps, and encourages the use of plain data and functions first.
Fair enough---I guess part of my dissonance is that's also a lisp, and I feel like 99% of the people attracted to lisps are us wild-eyed hobbyist types who really just want to build an individualized DSL for every project.
The s-expressions in lisps and clojure are one of my favorite attractions. There's so much less cognitive load when the syntax is so simple. Elixir, another great language, has a painfully complicated syntax (plus a lot of syntactic sugar and cases of there's more than one way to do something).
Clojure is a bit noisier than lisp in terms of syntax, but it's also more expressive. I find the balance and design choices pretty reasonable.
Using a functional language doesn’t make reality and the complexity of state go away (unless you are writing a compiler or a program/system that fully and purely mathematically controls its inputs and outputs).
A good functional language lets you clearly define, manage, and trace where your code meets reality, though.
That kind-of needs strict static type systems, rather than loosey-goosey dynamic tag checking, though.
That's IMO the main reason statically type checked languages like C++/Java/etc work much better at large scale than runtime tag checked languages like Python/Clojure/etc.
I think "needs strict static type systems" is a hypothesis but one with certainly many anecdotal examples to refute it. Most of the studies that have been done (and admittedly these studies are extremely difficult to do well) show lower or similar bug counts in dynamic languages vs static languages.
If you've built large scale system with statically type checked languages that solve all these problems, I assume you never had any bugs right? Never needed to write any unit tests?
Yeah Python culture is generally pretty "direct". I did mostly python the last decade and got so used to "no problem to read the source of pytorch or whatever" that I meanwhile really feel challenged when I look at Java, C# whatever framework/library code to figure out where the abstractions end and the actual work is happening.
My own code also became less and less abstract over time.
I think it works well in a micro-service world, because in that world, no single component is large, as the large system is built of smaller services. So each service can be a small to medium sized Clojure code base. (At least it does for us at my work).
Apart from that, I think one thing that's missing for larger scale Clojure is best practices and common tools, IDEs, frameworks and libraries. When I compare it to Java, Java has so much ingrained best practice and information out there, even junior devs will quickly pick up a book about its design patterns, and all that. And basically everyone uses the same tools and frameworks, its always Spring or Guava, IntelliJ or Eclipse, Jetty or Netty, Log4J or Logback, Jackson, Hibernate, etc.
I might agree with you slightly on some of the ackward bits. If transducers had been there from the get go instead of lazy sequences. If named arguments were first class and the compiler could check that the mandatory arguments were passed in. If exceptions were functional, and pattern matching was added to compose them. And if a few more core functions/macros were added for convenience. I think it would facilitate a bit adoption on larger projects.
At the company I work for we've found it hard to find Clojure devs. Not sure if this applies everywhere else, but it's definitely seen as a disadvantage of the language here.
Many Clojure companies find it easy to hire smart non-Clojure developers and teach them Clojure. You can learn the basics in a week or two and be a pretty effective Clojure developer (especially if supported in a team) pretty easily.
I have helped do this at two companies that used Clojure heavily, and agree it is definitely doable (and possibly easier the smarter and more junior they are).
What can be more challenging is to find more senior people who are decent Clojurians (it is hard to take a junior developer and make them as, or more, senior than you, for example :-) ). Hiring experienced Clojure devs is also doable, but takes time, flexibility (e.g., open to remote... easier in 2020 than in 2019), and real effort.
I can coroborate this in our case. But to be fair to OP, it requires having a very good onboarding process. We have weekly Clojure lessons where we teach the language, and good CR protocol so for a while all CRs go to people who know the language very well so they can give good advice about Clojure. And this works if you add new developers slowly, but can get challenging if you hire quickly. So I would say it still requires some extra effort and the motivation to do so.
+1 on that. I've learned Clojure in the past and keep wanting to go back to it but Clojure has been the hardest language for me to become fairly comfortable with. I think it may have to do with its dynamic-ness. There's basically no problem Clojure experts can't find a solution for. Its basic constructs are very powerful but all those solutions are not codified in the language itself. So Clojure has this extra layer of "culture" around it and as a beginner that's very hard to learn without a lot of doing and running into a wall quite a few times. Just when I think I have a grasp on the language, I see a new construct or new way of doing something that's completely alien to me and wasn't in the Clojure book I just read. I've not saying one can't become good at it but it takes a bit and might help to have an expert or two guiding you.
As a programmer with decades of experience, it took me half a year to get Clojure, and I had to unlearn a lot. Reading books is not enough, one has to do programming exercises with the language.
So I agree that it is harder for experienced people than for young kids fresh from school.
Yes, you can, if the new hire is fresh from school and is smart.
In my startup, our onboarding time is two weeks. I would say nine out of ten fresh graduates we hired picked up Clojure and were able to do something useful after two weeks. But we did have one or two persons who were not able to and they ended up in one of FAAG (yes, we did do leetcode style interviews, but only asked medium level questions).
Depends on where they're coming from.
If all they know is Java, then surely not!
If they've done Python and C and Erlang before, the switch doesn't seem as hard.
I recently inherited a Clojure codebase whose original developers are all long gone. I'm finding the experience to be a bit like trying to learn how to navigate a city where none of the streets have names.
When I was working my way through Clojure for the Brave and True, I really liked all the clever techniques around data-level programming and informal interfaces. Not having to stop to define and name formal types lets you slap things together quickly. But I'm now seeing the where that approach leads to: A codebase where nothing is formally defined, and nothing has a name.
I see where there were some efforts to do it by the book, and formalize this stuff by using functions to define an interface. The problem, though, is that we live in an orderly cosmos, and some laws are universal. One of those laws is that people will always take the shortest practical path to get from point A to point B. No matter how politely you ask them not to. So, if a map can be manually hacked to shreds in an ad-hoc manner, it will be manually hacked to shreds in an ad-hoc manner.
I think I would find it great fun to take a Clojure codebase like this one that you inherited and refactor it to something workable and maintainable! I agree it's probably not a job for a Clojure newbie.
The idea that you need "wise programmers" is a really big problem. People leave, institutional knowledge is lost, and then the code archaeology is just more difficult than it would be where types exist as guard rails.
I think Clojure looks favorable in comparison to Java still (though perhaps less so). But with Kotlin and Scala as mature alternatives -- and Clojure without a real niche -- it's not what I would reach for.