System.Text.Json.JsonException: read too much or not enough

Recently, I needed to create a custom json value converter to read nested value of a json property on deserialization.

E.g. given this structure

“primaryPosition”:{
“code”:”8",
“name”:”Outfielder”,
“type”:”Outfielder”,
“abbreviation”:”CF”
}

I would like to read “abbreviation” property and map it to a POCO property. So, I went ahead and created custom JSON value converter to do that

This converter then will be applied to a property

I run tests to see this change in action, but I got weird JsonException:

The converterValueConverters.PositionConverter’ read too much or not enough. Path: $.primaryPosition | LineNumber: 7 | BytePositionInLine: 24

Huh ? what does it mean ?

I went back to Microsoft documentation on custom converter creation to see if I missed anything.

Specifically Steps to follow the basic pattern section doesn’t mention of anything that I don’t have.

At this point I decided to look into Github to find System.Text.Json.JsonConverter<T> implementation and see what is going on.

JsonConverterOfT.VerifyRead

I found JsonConverterOfT.cs on Github and started to go through the code.

Read code execution eventually ends up calling VerifyRead method, which checks

• If original TokenType (TokenType of the Utf8JsonReader upon entering Read method) was StartArray, then current TokenType is EndArray & CurrentDepth is equal the original Depth (value of CurrentDepth property of the Utf8JsonReader upon entering Read method), otherwise throw JsonException(“read too much or not enough”)
• If original TokenType (TokenType of the Utf8JsonReader upon entering Read method) was StartObject, then current TokenType is EndObject & CurrentDepth is equal the original Depth(value of CurrentDepth property of the Utf8JsonReader upon entering Read method), otherwise throw JsonException(“read too much or not enough”)
• Default case (let’s skip over it)

The exception we saw makes sense now. We return as soon as reading value of abbreviation property, leaving reader’s current Token on PropertyName token, never reaching the EndObject token (Since original TokenType was StartObject).

Fix would be to simply continue reading until Utf8JsonReader reaches EndObject token:

And indeed running tests again are successfully deserializing Position property and the JsonException is gone.

However, this works because depth of StartObject & EndObject tokens are the same. As we’ve seen in VerifyRead method, original & current depth equality is also checked. Let’s try to see this with different JSON structure. Suppose we have more nested json structure

Our goal is to map dexterity.batSide.description to a POCO property. If we create custom JsonConverter and use same approach to continue reading on Utf8JsonReader until it reaches EndObject token, we’d get same JsonException.

If we debug BatSideConverter, we can see that while reader’s token type is EndObject, current depth differs from original depth (depth upon entering Read method). And as we’ve seen above, VerifyRead implementation also verifies original/current depth equality.

Considering this, correct implementation would be to Read until depths are equal & TokenType is equal to EndObject.

I published code to GitHub in case you’d like to see this in action.

P. S. For what it’s worth, I submitted an issue to microsoft docs suggesting to mention requirements for Utf8JsonReader’s TokenType & CurrentDepth properties when creating a custom value converter.