encoding/json: Marshaler/Unmarshaler not stream friendly

[lukescott]

The Marshaler/Unmarshaler interface deals with whole []byte slices:

type Marshaler interface {
        MarshalJSON() ([]byte, error)
}
type Unmarshaler interface {
        UnmarshalJSON([]byte) error
}

If you're dealing with a type that encodes to an array that has a large number of objects inside you have to encode all of into a single []byte.

The encoding/xml package is, unlike encoding/json, stream friendly:

type Marshaler interface {
        MarshalXML(e *Encoder, start StartElement) error
}
type Unmarshaler interface {
        UnmarshalXML(d *Decoder, start StartElement) error
}

With MarshalXML you can call e.Encode/e.EncodeElement/e.EncodeToken.

Since encoding/json is gaining Token()/EncodeToken() methods it would be really helpful to have a Marshaler/Unmarshaler interface that can take advantage of that. Perhaps something along the lines of:

type Encoder interface {
        EncodeJSON(e *Encoder) error
}
type Decoder interface {
        DecodeJSON(d *Decoder) error
}

Since Marshaler/Unmarshaler can't be changed. Or something like:

type Marshaler2 interface {
        MarshalJSON(e *Encoder) error
}
type Unmarshaler2 interface {
        UnmarshalJSON(d *Decoder) error
}

[mildred]

This would be really nice to have. I looked a bit into the code and found that it won't be easy to use the Unmarshaler2 interface while just using the json.Unmarshal(...) function, because we don't have a Decoder object.

However, it's more easy to have this implemented when using Decoder.Unmarshal(...)

[JonathanFraser]

I think it's worth considering looking at the way the yaml package handles custom unmarshallers.

type Unmarshaller interface {
    UnmarshalYAML(unmarshal func(interface{}) error) error
}

type Marshaller interface {
    MarshalYAML() (interface{}, error)
}

While I would normally dislike empty interfaces when it comes to marshalling its kind of unavoidable. What I like about this is it allows the object so simply inject the marshallable type it wishes to use under the hood without exposing a raw byte string. This has a nice side effect of decoupling the encoding from the object. For example, rather than this be a MarshalYAML and UnmarshalYAML interface this could just be Marshal and Unmarshal. This is nice because it breaks dependancies, moreover, the user just needs to specify the underlying marshalling type and it doesn't really care if it's decoded as a stream or a buffer. It's completely agnostic to that.

[smyrman]

The linked issue #14750 is not dependent on or directly related to this issue. It just raises another potential use-case for the described interface.

The use-case is relevant for the described Decoder interface (ignoring the naming conflict) in particular. By passing along a (sub) json.Decoder instead of []byte to a DecodeJSON method, you could also pass along options set on the original decoder, such as UseNumber, DissalowUnknownFields etc. If it's indeed a sub-decoder/copy, then it should be possible to override these properties without affecting the parrent. E.g.

const jsonExample = `{"a": "foo", "b": "bar", "c": 42}`

type T struct {
   A string
   B string
   C interface{}
}

func (t *T) DecodeJSON(dec json.Decoder) error {
   // Rely on parent decoder option for DissalowUnknownFields.
    dec.UseNumber() // Override UseNumber for sub-decoder only.
    return dec.Decode(t)
}

[mvdan]

For the sake of deduplicating encoding/json issues, I'm going to close this issue in favor of #7872 and #11046. There's also #33714 with a specific design and implementation.