[Server-side Kotlin] Using Moshi for Ktor serializer
Introduction
Hello. My name is Shiode, and I'm in charge of payment-related back-end development at the Woven Payment Solution Group.
Our group uses Kotlin for development, and we use Ktor as the web framework. Ktor requires a separate JSON library to process JSON request bodies and return JSON responses. At the beginning, we used Gson as a JSON library, but we encountered problems using Gson, so we switched to Moshi.
Today, I'm going to describe why we chose Moshi, the problems we faced when we used Moshi with Ktor, and how we solved them.
The version of Ktor used in this article is the 2.x series.
Switching from Gson to Moshi
Issues with Gson
Initially, we were using Gson as a Ktor JSON library. However, when receiving request bodies, we discovered that problems would occur if a necessary key was missing.
For example, think about an endpoint that processes input as follows.
data class Input (
val required: String,
val optional: String? = null,
)
post("/") {
val input = call.receive<Input>()
call.respondText("required was ${input.required}", status = HttpStatusCode.OK)
}
Since String
type is required
for the Input
class, null is not permitted. However, null is permitted with optional
. This endpoint is a simple one that receives the above Input
as a request body and returns the required
of the received body.
With this endpoint, a NullPointerException (commonly known as Nullpo) will occur if you send an empty JSON {}
that does not have a required
. It would be preferable for an exception to occur during deserialization so as not to affect later processing, but an exception ends up occurring when accessing that variable, in this case call.respondText()
.
As a solution, there may be a way to make the Input class required
a nullable type and perform a null check. However, since we're using Kotlin, we wanted to express it as a type where nulls are not allowed as far as possible, and write processing that is suitable for Kotlin.
Therefore, it became necessary to select a JSON library that would throw an exception during deserialization.
Why Did We Choose Moshi?
This project is developed with a schema base that uses OpenAPI, and the generation of interacting classes depends on the OpenAPI Generator. Also, since the client also uses the one generated by the OpenAPI Generator, a JSON library supported by OpenAPI was brought up as a replacement candidate.
The JSON libraries supported by OpenAPI Generator are shown below. Gson is excluded.
Jackson can support null safe by enabling Option, but when we looked at the relevant source code, performance was described as being affected. We decided to eliminate it as a candidate without comparing performance.
Meanwhile, kotlinx.serialization is a library developed by JetBrains, and the fact that it's native to Kotlin and compatible with Ktor made it is a good candidate. However, Moshi has twice as many stars as kotlinx.serialization (at time of writing, Moshi: 8.5K, kotlinx.serialization: 4.1K), and Moshi is Gson v3, so we decided to adopt Moshi.
Problems Using Moshi with Ktor
According to Ktor's PR#2004, there is no Ktor support for Moshi at the time of writing. Also, according to PR#2005, there seems to be no policy to provide Moshi as a core feature of Ktor in the future, either.
On the other hand, it seems that Ktor is planning to provide a Marketplace, so there is a possibility that in the future a third party will provide a Ktor plugin that will allow Moshi to be used. But there is no support for it right now, so you'll have to make your own Ktor plugin to be able to use Moshi.
Note that there is no information yet about the Marketplace at the time of writing. We're looking forward to hearing more about it.
Solution
Create a Ktor Plug-in to Enable Use of Moshi
The following are the classes required for using Moshi with Ktor. This implementation is almost the same as GsonConverter for Ktor.
class MoshiConverter(
private val moshi: Moshi = Moshi::Builder().build(),
) : ContentConverter {
override suspend fun serialize(
contentType: ContentType,
charset: Charset,
typeInfo: TypeInfo,
value: Any
): OutgoingContent? {
return TextContent(
moshi.adapter(value.javaClass).toJson(value),
contentType.withCharsetIfNeeded(charset)
)
}
override suspend fun deserialize(charset: Charset, typeInfo: TypeInfo, content: ByteReadChannel): Any? {
return withContext(Dispatchers.IO) {
val body = content.toInputStream().reader(charset).buffered().use { it.readText() }
moshi.adapter(typeInfo.type.java).fromJson(body)
}
}
}
It is used as shown below.
install(ContentNegotiation) {
register(ContentType.Application.Json, MoshiConverter())
}
Compatibility with Moshi ArrayList
When we use the above Ktor plugin, it deserializes as a JAVA type instead of a Kotlin type when deserializing. For example, it will deserialize as an ArrayList
even though you want to store it in a List
. Moshi doesn't support ArrayList
deserialization by default, so an exception will be thrown when you try to deserialize a List
. To support any type in Moshi, you need to prepare an Adapter for that type. Since we want to support ArrayList
this time, we must prepare a Moshi adapter for ArrayList
ourselves.
The following adapters support ArrayList
. This is almost identical to Moshi's CollectionJsonAdapter implementation.
abstract class ArrayListJsonAdapter<C : MutableCollection<T?>, T> private constructor(
private val elementAdapter: JsonAdapter<T>
) : JsonAdapter<C>() {
abstract fun newCollection(): C
override fun fromJson(reader: JsonReader): C {
val result = newCollection()
reader.beginArray()
while (reader.hasNext()) {
result.add(elementAdapter.fromJson(reader)!!)
}
reader.endArray()
return result
}
override fun toJson(writer: JsonWriter, value: C?) {
writer.beginArray()
for (element in value!!) {
elementAdapter.toJson(writer, element)
}
writer.endArray()
}
override fun toString(): String {
return "$elementAdapter.collection()"
}
companion object Factory : JsonAdapter.Factory {
override fun create(type: Type, annotations: Set<Annotation>, moshi: Moshi): JsonAdapter<*>? {
if (annotations.isNotEmpty()) return null
return when (type.rawType) {
ArrayList::class.java -> {
newArrayListAdapter<Any>(type, moshi).nullSafe()
}
Set::class.java -> {
newLinkedHashSetAdapter<Any>(type, moshi).nullSafe()
}
else -> null
}
}
private fun <T> newArrayListAdapter(type: Type, moshi: Moshi): JsonAdapter<MutableCollection<T?>> {
val elementType = Types.collectionElementType(type, Collection::class.java)
val elementAdapter = moshi.adapter<T>(elementType)
return object : ArrayListJsonAdapter<MutableCollection<T?>, T>(elementAdapter) {
override fun newCollection(): MutableCollection<T?> = ArrayList()
}
}
private fun <T> newLinkedHashSetAdapter(type: Type, moshi: Moshi): JsonAdapter<MutableSet<T?>> {
val elementType = Types.collectionElementType(type, Collection::class.java)
val elementAdapter = moshi.adapter<T>(elementType)
return object : ArrayListJsonAdapter<MutableSet<T?>, T>(elementAdapter) {
override fun newCollection(): MutableSet<T?> = LinkedHashSet()
}
}
}
}
Verifying Operation
After creating a Ktor plug-in for using Moshi with Ktor and a Moshi adapter for deserializing ArrayList
with Moshi, the next step is to verify operation. In order to use the one I just described, we need to write the following
install(ContentNegotiation) {
register(ContentType.Application.Json, MoshiConverter( moshi = Moshi::Builder().add(ArrayListJsonAdapter.Factory)
))
}
Then, let's try sending an empty JSON {}
to the endpoint introduced in the first example in this article. Here's that endpoint again.
data class Input (
val required: String,
val optional: String? = null,
)
post("/") {
val input = call.receive<Input>()
call.respondText("required was ${input.required}", status = HttpStatusCode.OK)
}
At first, nullpo was generated when accessing required
, but now the following exception occurs for call.receive<Input>()
during deserialization, as expected.
com.squareup.moshi.JsonDataException: Required value ‘required’ missing at $
Conclusions
- Nullpo can occur when using Gson with Ktor.
- If you don't want nullpo to occur, you should choose another JSON library.
- To use Moshi with Ktor, you need to create your own Ktor plugin by yourself.
- If you do that, you need to make a Moshi adapter to deserialize
ArrayList
.
- If you do that, you need to make a Moshi adapter to deserialize
References
関連記事 | Related Posts
We are hiring!
【Woven City決済プラットフォーム構築 PoC担当バックエンドエンジニア(シニアクラス)】/Toyota Woven City Payment Solution開発G/東京
Toyota Woven City Payment Solution開発グループについて私たちのグループはトヨタグループが取り組むWoven Cityプロジェクトの一部として、街の中で利用される決済システムの構築を行います。Woven Cityは未来の生活を実験するためのテストコースとしての街です。
【Toyota Woven City決済プラットフォームフロントエンドエンジニア(Web/Mobile)】/Toyota Woven City Payment Solution開発G/東京
Toyota Woven City Payment Solution開発グループについて我々のグループはトヨタグループが取り組むToyota Woven Cityプロジェクトの一部として、街の中で利用される決済システムの構築を行います。Toyota Woven Cityは未来の生活を実験するためのテストコースとしての街です。