Distinguish between NULL and not present using JSON Merge Patch with NetCore WebApi and System.Text.Json

I want to support partial updates with JSON Merge Patch. The domain model is based on the always valid concept and has no public setters. Therefore I can't just apply the changes to the class types. I need to translate the changes to the specific commands.

Since classes have nullable properties I need to be able to distinguish between properties set to null and not provided.

I'm aware of JSON Patch. I could use patch.JsonPatchDocument.Operations to go through the list of changes. JSON Patch is just verbose and more difficult for the client. JSON Patch requires to use Newtonsoft.Json (Microsoft states an option to change Startup.ConfigureServices to only use Newtonsoft.Json for JSON Patch (https://learn.microsoft.com/en-us/aspnet/core/web-api/jsonpatch?view=aspnetcore-6.0).

Newtonsoft supports IsSpecified-Properties that can be used as a solution for JSON Merge Patch in the DTO classes (How to make Json.NET set IsSpecified properties for properties with complex values?). This would solve the problem, but again requires Newtonsoft. System.Text.Json does not support this feature. There is an open issue for 2 years (https://github.com/dotnet/runtime/issues/40395), but nothing to expect.

There is a post that describes a solution with a custom JsonConverter for Web API (https://github.com/dotnet/runtime/issues/40395). Would this solution still be usable for NetCore?

I was wondering if there is an option to access the raw json or a json object inside the controller method after the DTO object was filled. Then I could manually check if a property was set. Web Api closes the stream, so I can't access the body anymore. It seems there are ways to change that behavior (https://gunnarpeipman.com/aspnet-core-request-body/#comments). It seems quite complicated and feels like a gun that is too big. I also don't understand what changes were made for NetCore 6.

I'm surpised that such a basic problem needs one to jump through so many loops. Is there an easy way to accomplish my goal with System.Text.Json and NetCore 6? Are there other options? Would using Newtonsoft have any other bad side effects?

With the helpful comments of jhmckimm I found Custom JSON serializer for optional property with System.Text.Json. DBC shows a fantastic solution using Text.Json and Optional<T>. This should be in the Microsoft docs!

In Startup I added:

services.AddControllers()
  .AddJsonOptions(o => o.JsonSerializerOptions.DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingDefault)
  .AddJsonOptions(o => o.JsonSerializerOptions.Converters.Add(new OptionalConverter()));

Since we use <Nullable>enable</Nullable> and <WarningsAsErrors>nullable</WarningsAsErrors> I adapted the code for nullables.

public readonly struct Optional<T>
    {
        public Optional(T? value)
        {
            this.HasValue = true;
            this.Value = value;
        }

        public bool HasValue { get; }
        public T? Value { get; }
        public static implicit operator Optional<T>(T value) => new Optional<T>(value);
        public override string ToString() => this.HasValue ? (this.Value?.ToString() ?? "null") : "unspecified";
    }

public class OptionalConverter : JsonConverterFactory
    {
        public override bool CanConvert(Type typeToConvert)
        {
            if (!typeToConvert.IsGenericType) { return false; }
            if (typeToConvert.GetGenericTypeDefinition() != typeof(Optional<>)) { return false; }
            return true;
        }

        public override JsonConverter CreateConverter(Type typeToConvert, JsonSerializerOptions options)
        {
            Type valueType = typeToConvert.GetGenericArguments()[0];

            return (JsonConverter)Activator.CreateInstance(
                type: typeof(OptionalConverterInner<>).MakeGenericType(new Type[] { valueType }),
                bindingAttr: BindingFlags.Instance | BindingFlags.Public,
                binder: null,
                args: null,
                culture: null
            )!;
        }

        private class OptionalConverterInner<T> : JsonConverter<Optional<T>>
        {
            public override Optional<T> Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
            {
                T? value = JsonSerializer.Deserialize<T>(ref reader, options);
                return new Optional<T>(value);
            }

            public override void Write(Utf8JsonWriter writer, Optional<T> value, JsonSerializerOptions options) =>
                JsonSerializer.Serialize(writer, value.Value, options);
        }
    }

My test DTO looks like this:

public class PatchGroupDTO
    {
        public Optional<Guid?> SalesGroupId { get; init; }

        public Optional<Guid?> AccountId { get; init; }

        public Optional<string?> Name { get; init; }

        public Optional<DateTime?> Start { get; init; }

        public Optional<DateTime?> End { get; init; }
    }

I can now access the fields and check with .HasValue if the value was set. It also works for writing and allows us to stripe fields based on permission.