Why we moved from Kotlin & Spring Boot to Go
Our product completely supports Java and we love Java developers. However, internally for our own code, we believe the era of the JVM is coming to an end.
We found these 3 things initially attractive about Go:
- Instant compile times and program startup for faster development
- Virtual threads for dealing with high concurrency workloads
- Value types to better control memory usage
We found a ton more to like later, but this was enough to give it a go.
Hurdles
Right off the bat, Go's compile-time enforced formatting rules and casing of variables seemed like a weird choice. However, soon after I warmed up to them knowing that they enforced a certain amount of code uniformity across our codebase.
Why does Golang use identifer capitalization for public/private visibility? This seemed like an insane design decision to me. Now I realize. Go is simply saying 'Naming conventions are not guidelines, they are rules'. Uniform naming conventions across 100% of Go code is good.
— Prashant Deva (@pdeva) January 5, 2021
The biggest hurdle though was the lack of an equivalent of a Spring Framework for Go. While the Go ecosystem has multiple libraries for all kinds of functionality, there didn't seem to be a standard way of doing things like logging, dependency injection, environment configuration, etc. We had to solve this before we could adopt Go in a major way in our organization. Thus, alongside developing the first few microservices, we kept working on an in-house framework, Gallium. The Gallium Framework could use a blog post of its own. In short, it standardizes on some libraries in the Go ecosystem for things like Logging, DI, etc, and makes it easy to structure microservices for our environment. We keep evolving Gallium with learnings from each Go microservice we develop.
We do miss JOOQ though. goqu
comes close, but its not the same.
I would gladly pay for a version of @JavaOOQ that generated Golang code
— Prashant Deva (@pdeva) December 26, 2020
Also, the time
package could use some improvements:
Golang not parsing ISO8601 Durations seems like a weird omission
— Prashant Deva (@pdeva) January 2, 2021
Though, its still a lot better than the java.time
insanity:
Java 8's 'new' Time API:
— Prashant Deva (@pdeva) August 20, 2019
Wanna store time? Pick from
1. Instant
2. LocalDateTime
3. OffsetDateTime
4. ZonedDateTime
5. LocalTime
6. OffsetTime
7. LocalDate
Wanna store ISO Period? Pick from
1. Period
2. Duration
LuxonJS OTOH:
- DateTime
- Duration
Does work of all 9 above
In the land of magic
We took one of our simpler but high throughout Spring Boot service called metric-query
and converted it to Go. metric-query
takes in queries from our dashboard and runs them against our metric database. Since our dashboards refresh every 2 seconds, it can get a lot of requests at once. In the JVM world, each of these requests would spawn a whole thread.
After converting to Golang, we saw the memory usage drop by 40x! We had to double-check that our charts were not faulty.
Converted a high throughput service from Kotlin & Spring Boot to Golang. Memory usage went down from ~800MB per instance to ~20MB per instance!! pic.twitter.com/KATZenY8OG
— Prashant Deva (@pdeva) January 3, 2021
This convinced us we were on the right path and the investment in Go was worth it.
We then converted our highest throughout service, metric-collector
. It ingests data from all our agents and puts it on Kafka. It receives a ton of requests all at the same time. The Spring Boot version was maxing the CPU on the nodes it was deployed and eating up tons of memory. Latency was all over the place.
Converting it to Go and watching the graphs was pure magic. CPU usage and latency dropped to almost nothing. Memory usage dropped to insignificant levels. This could be a case study for virtual threads.
Kafka producer latency after switching ultra high throughput app from Kotlin to Golang. The latency graph almost feels like it disappears after the switch. pic.twitter.com/kewMpkh4tQ
— Prashant Deva (@pdeva) January 9, 2021
Finished migrating the ultra high throughput service from Kotlin to Golang. Memory usage went from 3.1GB to 125MB! pic.twitter.com/ioVLssxnMz
— Prashant Deva (@pdeva) January 10, 2021
We have continued converting each microservice from Kotlin/Spring Boot to Go and its magical each time watching those graphs drop.
Cant get enough of seeing the graphs change as I switch a service from Kotlin to Go pic.twitter.com/kS2nNPRv29
— Prashant Deva (@pdeva) January 12, 2021
Looking back at JVM world
I have been programming on the JVM for almost 2 decades. Switching to Go was quite a change for me personally too.
But looking back, the Java/Spring world seems to be toppling to maintain compatibility with all its legacy. It is mind-boggling to think a microservice in 2021 still needs Tomcat to run. Spring MVC is layers and layers of code to get Java Servlets, which were released in 1996, somehow feel relevant in 2021. Kotlin is another massive layer to make Java feel like a modern language.
I remember writing this Spring MVC code and thinking it was so clean. Now that my blinders are off, it feels like a messy DSL built on annotations. No way to step through flow of logic. To do anything, I need to find appropriate annotation by digging through docs. Its a black box pic.twitter.com/onwwkx4mjk
— Prashant Deva (@pdeva) January 14, 2021
The Maven/Gradle build system seems like a ridiculous, unnecessary complexity compared to Go's dependency management.
It is so liberating to be free of the complexity of Maven/Gradle/NPM/Webpack.
— Prashant Deva (@pdeva) January 8, 2021
All one really needs to build a project is a text files with dependencies listed on each line. I just do a `go build` and it produces a binary. I dont need to read an entire book to build my project. pic.twitter.com/MySfY6p8jv
The JVM, Java, and Spring were great for an era indeed. In 2021, they feel like an 80-yr old trying to compete in the Olympics by taking artificial performance enhancing drugs. Switching to Go feels like a refreshing, exciting change.
converting a service from Kotlin/SpringBoot to Golang feels like stepping into the future.
— Prashant Deva (@pdeva) December 28, 2020
- Instant compilation
- Instant startup
- No special code to make async queries
- No worrying about too many threads
I encourage everyone to try it