Support a default value for non-nullable properties

[thomaslevesque]

Suppose I have an entity like this:

public class Todo
{
    public string Id { get; set; }
    public string Title { get; set; }
    public bool IsDone { get; set; }
}

If the container has a document with no value defined for IsDone, trying to load it throws an InvalidOperationException: Nullable object must have a value.

Although I understand the reason for this, it means my code can never handle documents where IsDone is null or undefined (which can happen, for instance, if the property didn't exist in the beginning but was added later). I could make the property nullable in the entity, but then I would need to handle the null case everywhere the property is used, which isn't ideal.

Also, if I query with a filter like .Where(t => !t.IsDone), documents where the IsDone property is undefined are not returned. I understand that it's because of the Cosmos DB semantics (null or undefined is neither true nor false), but it would be nice if null or undefined could be handled as if it was a default value (false in this case). See also Azure/azure-cosmos-dotnet-v3#682.

I think this could easily be handled by supporting a default value. This value could either be specified with a [DefaultValue] attribute on the property, or with a HasDefaultValue(T value) method in the fluent model builder API. The effect would be to change the SQL from NOT(doc['IsDone']) to NOT(doc['IsDone'] ?? <defaultValue>).

I think it would be a very valuable feature to make data model evolutions easier. Currently, if I want to add a property when the container already contains documents, I have to either make it nullable (which isn't ideal if it's not supposed to be null) or update all documents to add the property.

[ajcvickers]

@AndriySvyryd Duplicate of #13131, but this was split into dotnet/EntityFramework.Docs#1712, #17722, and #17746. I'm not which of these matches this particular request.

[AndriySvyryd]

@ajcvickers We can keep this one separate then as none of the split ones match it.

[steveevers]

This is very much needed, however, you can get most of the way there if you don't need to query using the property by doing something like this:

public class Todo
{
    public string Id { get; set; }
    public string Title { get; set; }
    public bool? isDone { get; set; }

    [NotMapped]
    public bool IsDone
    {
        get { return this.isDone.HasValue && this.isDone.Value; }
        set { this.isDone == value; }
    }
}

That is to say, make it nullable, but make a wrapper around it that defaults the null to false when read. You get the safety of having a default value for when the document doesn't contain the value and you can still deserialize the type from cosmos without throwing exceptions.

[vyarymovych]

@steveevers In general it works, but there is one caveat.
It doesn't work if you want to apply query projection which is very important with regards to query performance (to query specific props rather than everything). In this case it doesn't even execute property getter and crashes with the same error ('Nullable object must have a value.'). Example: .Select(t => new TodoProjection { IsDone = t.IsDone })
Any ideas how to workaround this?

[dbalikhin]

Any update on this feature?
It's very annoying to add new non-nullable fields.

[ajcvickers]

@dbalikhin This issue is in the Backlog milestone. This means that it is not planned for the next release (EF Core 7.0). We will re-assess the backlog following the this release and consider this item at that time. However, keep in mind that there are many other high priority features with which it will be competing for resources. Make sure to vote (👍) for this issue if it is important to you.

[dbalikhin]

Thanks @ajcvickers.
I don't think EF Core is even usable with CosmosDB without this feature. Unless you don't need to add new fields in the future. But that's the point of unstructured data storage.
New fields defined as nullable: cannot properly use a.HasValue / a.Value in query, because of null vs undefined.
New fields defined as non-nullable: cannot properly retrieve items, because this field can be undefined.
No migrations to set the default value on field creation.
More like CosmosDB limitations of course.

[dbalikhin]

Okay, so it doesn't like nullable.HasValue and nullable.Value, but it works with "nullable.FieldName condition X".
So nullable.IsDone == true or nullable.IsDone == null.
Stick with nullable for now...

[rcmosher]

One option for providing a default value is to use a ValueConverter with convertsNulls set to true. An example for a boolean that defaults to false when undefined or null.

  public class UndefinedDefaultConverter : ValueConverter<bool, bool?>
  {
#pragma warning disable EF1001 // Internal EF Core API usage.
    public UndefinedDefaultConverter() : base(
      v => v,
      v => v ?? false,
      convertsNulls: true
    )
#pragma warning restore EF1001 // Internal EF Core API usage.
    { }
  }

Note, this doesn't help with querying since IsDefined() isn't handled by EF. I haven't found a way other than writing your own query to properly check for IS_DEFINED(), null, or the default value.

[mnguyen36]

@AndriySvyryd #13131 doesn't really explain what EF core does or should do to support this use case (supporting non-nullable values by allowing defaults)
#17722 and #17746 also don't directly enable this.

Is there not a simple way to allow EFCore to let the natural c# way of initializing defaults occur without error?
i.e.

public class Todo
{
    public string Id { get; set; }
    public string Title { get; set; }
    public bool IsDone { get; set; } = false; 
}

In the case where you need to query by these values i.e. _context.ToDo.Where(td => td.IsDone), any values who aren't there should be treated just as cosmos handles it, but itself would be a separate responsibility than the one saying that you can't have nulls. I was looking at adding the IS_DEFINED function into EF.Functions but still can't connect how this will actually enable the big problem of EFCore not handling gracefully new values