I noticed last week or so that Apache Kafka deprecated large amount of Scala API and put new Java API in its place. This can't be good for languages which already generate so much news but so little software.
Few days back I start investigating Kafka for our project. I was under the impression that it was all Scala. But downloading, setting up, and reading document I realized that a large amount of Scala API is deprecated and replaced with newer Java equivalents.
Maybe it was. Currently Kafka codebase has twice the Java code than Scala. Over time I have seen Scala code keeps shrinking and Java code keeps rising in repo.
That's funny, when I talked to Neha (ex-Linkedin, cofounder of Confluent) she explicitly denied this meme about moving away from Scala.
The new Kafka client library is in Java just to reduce the number of dependencies, the server is still written in Scala. Strangely, using Java hasn't stopped Confluent from making breaking changes on dot release of the client... which really puts the lie to your explanation.
Even if some random kafka committer that you can't source believes that, the reasoning doesn't add up:
SBT makes it fairly straightforward to cross publish for different dot releases of Scala, and libraries in the Scala ecosystem do it all the time.
Scala dot releases are far apart. It's been over 2 years since 2.11, and 2.10 released in 2012.
By contrast, the JAVA client that the Kafka project published in November is already undergoing breaking changes for the current release candidate. It's pretty clear that avoiding breaking changes is not an overriding concern for them, and even if it is, it's equally clear that Scala was not the inherent problem.
> (Spark, Mesos, Akka, Cassandra, Kafka), most of which has been written in Scala.
Regarding Kafka, I noticed a large amount of Scala API is deprecated and replaced with Java API in the latest version. There is about twice the Java code in Kafka as compared to Scala.
Mesos is C++.
Cassandra is Java.
So it seems to me Spark and Akka is mainly Scala. And in this field I am expecting Rust to make big inroads with high performance, no JVM like overhead for memory safety, and lately emphasis on async IO with tokio.
Pretty sure this isn't even language specific. If you are using a library that drastically modifies how they implement something between versions, then you will run into this. Even in pure Java. Now, the ecosystem in Java is to usually not make large changes. At least not at the public API level.
I think Scala is slowing down in this area, but have not stopped.
As someone with a mvn-built Spark Streaming to Delta Lake project in Scala open in IDEA right this moment I find this article hugely depressing. I love every single technology I just mentioned but the sheer proposed scope makes me think it ain't gonna happen. The momentum is lost completely.
* IDEA works for Scala just fine but I wouldn't object to more refactoring features ported from the Java world
* the Akka debacle was enormous and made me think Scala is not viable anymore outside of the Spark/Flink/Kafka circles
* mvn needs a newer frontend/DSL but it's light years ahead of the SBTs/Gradles of the world otherwise
* WAScalaM seems to be the only alternative platform that is forward-looking enough to make sense
* FP has its place but the multi-year emphasis on it has damaged real-life adoption disproportionately
Scala isn't perfect, but it's a great language to have on the JVM.
However, the lack of API and ABI compatibility is a really big failure. I can't believe that they're shipping what is supposed to be stable software and we can't rely on our code to continue running, much less building.
We had to do a fair bit of work to fix our projects' 2.8 compatibility issues, and it's causing us all sorts of grief in terms of having to do lock-step upgrades of all libraries from 2.7 to 2.8.
It's also a real pain to call Scala code from Java, which makes blending Java and Scala code in a project, or adding Scala code to an existing Java project, much less attractive.
The newest version of Scala (2.12) requires a JVM with Java 8 feature compatibility which makes it unusable on Android devices. Google would literally be stuck supporting an obsolete version of a language that's not developed... or restrict the use of it only to Android O and newer making it dead on arrival.
Kotlin compiler generates code that's runnable on JVMs supporting Java 6, which makes it work on pretty much any Android device, even running older Androids.
Scala standard library is also huge and pretty much immediately hits the 64K method limit of DEX files forcing you to use the very slow proguard compilation step when running it. It also has issues with interoperability with existing Java 6 code.
All in all, Scala is a terrible value proposition in comparison to Kotlin which was built from ground up to modernize programming while still keeping full compatibility with Java 6 code and JVMs stuck on Java 6 language level.
reply