-
Notifications
You must be signed in to change notification settings - Fork 3.3k
Description
When querying against values coming out of a JSON document, we need to make sure that the JSON values get projected out as the correct store type, otherwise comparison with e.g. a regular column will fail. For SQL Server, most store types work by simply casting the string representation returned from OPENJSON; but with SQLite - which has very few types - there are some issues:
.NET type | Store representation | Default JSON representation | Notes |
---|---|---|---|
DateTime | 2023-01-01 12:30:00 | 2023-01-01T12:30:00 | Can be converted via datetime() |
DateTimeOffset | 2023-01-01 12:30:00+02:00 | 2023-01-01T12:30:00+02:00 | datetime() yields 2023-01-01 10:30:00 |
decimal | "1.0" (string) | 1 (number) | |
Guid | uppercase | lowercase | can use upper(), or store uppercase in JSON |
byte[] | X'0102' | AQI= (base64) | Same problem in SQL Server, though OPENJSON/WITH does decode base64 |
Options:
- Convert the JSON representation to the store type upon extraction; this is ideal, when supported. A CAST usually works for SQL Server, and in some cases a custom function is needed (e.g. datetime for SQLite). Adding the conversion function to the type mapping is part of Add type mapping APIs to customize JSON value serialization/deserialization #30677.
- Store the database's literal representation in JSON. In cases where there's a semi-standard JSON representation, this isn't great; for example, timestamps are commonly represented as ISO8601 in JSON, and storing another representation wouldn't be good for interoperability. The API for this would also be part of Add type mapping APIs to customize JSON value serialization/deserialization #30677.
- Leave the type as unqueryable - that is, storable via value converter only.
For option 1, we need type mapping API support which tells us how to convert values coming out of a JSON document into their equivalent relational store representation. Note that this is complicated somewhat in SQL Server, where in most cases we can simply use OPENJSON/WITH which automatically does the conversion for us, but in some cases we can't do that (because OPENJSON/WITH doesn't preserve ordering, and is also incompatible with geometry).