State stores: return a flag to indicate whether the returned value is empty or not present

[ItalyPaleAle]

Describe the feature

Currently, when a value is not found in a state store, components return an empty response (ie. with an empty data). Because of that, callers are not able to distinguish between an empty value stored in the database, or a missing value.

In the past this was "addressed" by making some state stores refuse to store values that were empty strings. This caused some issues, not least because it was an uneven behavior only in 4 components: #2722

A proper fix would be to add an extra value to the response object from the state stores' get method to indicate whether the value was found.

Such a change would be bacwkards-compatible, but needs to also be implemented in the runtime and possibly the SDKs.

Release Note

RELEASE NOTE: ADD Add flag to state store Get responses to distinguish empty values from missing ones

[berndverst]

Today our Get Response is define as follows:

type GetResponse struct {
	Data          []byte            `json:"data"`
	ETag          *string           `json:"etag,omitempty"`
	Metadata      map[string]string `json:"metadata"`
	ContentType   *string           `json:"contentType,omitempty"`
}

As discussed we do not have a way to indicate the difference between key not found and the key is empty.

Changing the method signature would be a breaking change we do not want to make, so adding a new property is the only way forward.

Which of the following two options would the community prefer?

1. isEmptyValue, a boolean to indicate whether the value was truly empty. Here empty data + isEmptyValue: false would mean that the key was not found.

2. isKeyNotFound, a boolean to indicate whether the key was found or not. Here isKeyNotFound: true and an empty data section (but really anything in the data section) would mean that the key is indeed not present.
   And empty data section + isKeyNotFound: false means that the key was the empty value instead.

From an implementation perspective, option (2) is less work and less error prone! We already have checks whether not results were returned - we only need to return the additional property in that case.

[berndverst]

I propose updating it to the following:

// GetResponse is the response object for getting state.
type GetResponse struct {
	Data          []byte            `json:"data"`
	ETag          *string           `json:"etag,omitempty"`
	Metadata      map[string]string `json:"metadata"`
	ContentType   *string           `json:"contentType,omitempty"`
	IsKeyNotFound bool              `json:"isKeyNotFound"`
}

Which means there will also need to be a corresponding property in the GetResponse proto.

All of this is additive and no behavior changes - we just provide clarification between the two flavors of empty values.