11.4.11 Position Inserts, allows inserts into, presumably ordered, collection of entity but limits $index query option to being a zero-based ordinal

[Hubert-Heijkers]

Section 11.4.11 Positional Inserts states that collections of entity, complex, or primitive types support inserting items at a specified location, yet only describes the use of $index for collections of complex or primitive types.

I'd like to be able to simply state that $index could be used to specify the key of the entity at the location at which the new entity ought to be inserted but realize that this wouldn't work for multi-part keys.

Section 11.4.10 Managing Members of an Ordered Collection might equally require an additional clarification. Whilst not explicitly mentioning collections of entity types, those are included as well but need no special treatment in this context as they are indexed by key.

PS #562, which is related, was closed as completed but couldn't find anything in the meeting minutes nor has the proposal be applied, seem to recall this issue came up in that discussion.

[ralfhandl]

PS #562, which is related, was closed as completed but couldn't find anything in the meeting minutes nor has the proposal be applied, seem to recall this issue came up in that discussion.

#562 has been applied, see 11.4.9.4 Update a Collection Property.

The cloned GitHub issue #562 points to the original Jira issue ODATA-1437, which in turn links to pull request #136 applying the changes. I've now directly cross-linked #562 and #136.

[ralfhandl]

I'd like to be able to simply state that $index could be used to specify the key of the entity at the location at which the new entity ought to be inserted but realize that this wouldn't work for multi-part keys.

A new entity can be inserted into a collection of entities by sending a POST request to the collection. This allows inserting fully keyed entities as well as entities where (part of) the key is server-generated.

The server can then decide which index position this new entity will have.

If the new index needs to be defined by the client, this could be done by annotating the new entity with a custom annotation such as @Custom.IndexDirectlyAfter: <list or map of key values>.

[Hubert-Heijkers]

I'd like to be able to simply state that $index could be used to specify the key of the entity at the location at which the new entity ought to be inserted but realize that this wouldn't work for multi-part keys.

A new entity can be inserted into a collection of entities by sending a POST request to the collection. This allows inserting fully keyed entities as well as entities where (part of) the key is server-generated.

The server can then decide which index position this new entity will have.

If the new index needs to be defined by the client, this could be done by annotating the new entity with a custom annotation such as @Custom.IndexDirectlyAfter: <list or map of key values>.

I wasn't suggesting to change how one adds an entity, nor would I like to change what goes into the body of the POST request (especially if I'm getting that from somewhere else - read: I'd therefore also prefer not to have to insert any [custom] annotations INTO the body either).

Reading the first sentence of 11.4.11, which explicitly also refers to/includes collections of entity types, stems me hopeful. Reading what follows in 11.4.11 leaves me wondering what to how I come up with that zero-based ordinal in case of a collection of entity type.
In such collection, in combination with optimistic concurrency, I could probably figure out the ordinal of an entity and then use that on the subsequent POST request, would you agree @ralfhandl?

So why not allow specifying the key or list/map of key values as the value for $index in case we are dealing with an ordered collection of entity?

[ralfhandl]

why not allow specifying the key or list/map of key values as the value for $index in case we are dealing with an ordered collection of entity?

A non-negative integer value of $index is where you expect the inserted member to be located at directly after the insert.

With a list/map of key values this is not the case, you expect it to be right after the numeric index of the entity whose key values you specified. Hence I'd prefer a new and precise name for this new feature.

[mikepizzo]

Having an ordered set of entities is a real scenario (i.e., steps in a pipeline). It would be good to define end-to-end support for such a method, including:

1. Insert into a position
2. Re-arrange the order of existing items

[mikepizzo]

Note that entities can be inserted ordinally using $index, but it requires knowing the index where you want to insert. If you just have the key that could require extra work to get the index.

This solves #1, but we don't currently define a way to re-arrange the order of existing items. SAP and IBM have defined actions for this. We could consider defining a canonical action for this, which required explicit annotation to apply to a particular collection.

Inserting an existing entity reference at a certain location could also be semantically interpreted as a move, assuming that collections of entities can't contain duplicates. However, there may be scenarios in which the same entity instance could existing at multiple locations within an ordered collection (which really isn't supported in OData today). In this case, it is really a collection of references to entities (but the user experience degrades if you have to wrap everything in a reference object).

If we did try to support having the same instance multiple times in an ordered collection, then there would be no way to reference the instance at one location versus the other; for example, to remove one occurrence and not the other. We could support this by relaxing the statement that you can't use $index for delete, but this would conflict with the syntax for deleting by key as segment (for integer keys). Heiko suggests this may be best supported by a PATCH on the collection (similar to JSON PATCH).

[mikepizzo]

For ordered collections, we could do DELETE collection/$index(1) (or collection/$index?@Ordinal={ordinal}). You would want to use an etag for the collection, with an if-match, to avoid accidentally deleting the wrong occurrence.

If we generalized this as general path syntax, then we could support the scenarios described in #342.

Introducing /$index({ordinal}) would introduce a potential ambiguity of someone had a key of $index({ordinal}), but we already have such ambiguities (i.e., with an action/function bound to the collection)

[mikepizzo]

This issue has really become more about how to support ordered collections of entities with duplicates.

@Hubert-Heijkers will do some homework to decide whether ordered collections of entities with duplicates is something that should be brought into the standard or is really just a modeling shortcut they have taken in their API.

[HeikoTheissen]

An ordered collection of entities with possible duplicates could be modelled in OData V4.02 as a collection of the following entity type:

<EntityType Name="OrderedMultiSet">
  <Key>
    <PropertyRef Name="Ordinal"/>
  </Key>
  <Property Name="Ordinal" Type="Edm.Int64" Nullable="false"/>
  <NavigationProperty Name="Reference" Type="Edm.EntityType"/>
</EntityType>

An entity set Nodes with this entity type could then be used to address entities ("nodes") by their ordinal number. Compare the following requests:

OrderedMultiSet approach (V4.02)	$index approach (V5.0)
GET Nodes(13)/Reference	GET Nodes/$index(13)
DELETE Nodes(13)	DELETE Nodes/$index(13)
POST Nodes(13) {"Reference":{"@id":...}}	POST Nodes/$index(13) {"@id":...}

The DELETE and POST requests would affect not only the entity at position 13 but also renumber all entities after that.