Last Modified Option Within API Calls about odd-platform

[Vladysl]

implemented in #1685

[RamanDamayeu]

Hi,

1. For terms we will work on adding "update_at" column (to follow the internal naming) to the results of this "/api/terms" endpoint for each item and also we will add additional parameters to the endpoint like "update_at_range_start_date_time" (optional) -- for the start of interval that includes value; and "update_at_range_end_date_time" (optional) - for the end of the interval that excludes the values. So the items would be with "update_at" >= update_at_range_start_date_time and "update_at" < update_at_range_end_date_time.
2. For data entities (and datasets in particular) we could suggest to use /activity/getActivity to track some (not all) modifications (this list of activities is tracked

   odd-platform/odd-platform-api/src/main/java/org/opendatadiscovery/oddplatform/dto/activity/ActivityEventTypeDto.java

   Line 3 in 5ca558b

   public enum ActivityEventTypeDto {

   ).
   Unfortunately, when we ingest data we do not track date of modification for most of the attributes (including metadata, description, etc.) the only part that is tracked is dataset structure: there is an endpoint /api/datasets/{data_entity_id}/structure (Get latest version's DataSet structure information) that has data_set_version.created_at attribute that shows when the version of the structure has been created. But I agree in that case we still need to look over all {data_entity_id} to find out what structures have been changed since some date in the past. So if getActivity doesn't cover the requirements it will be tough to create an additional attribute with the last modification date for datasets. We'll need to put a bit more effort to design the feature.

[clintjb]

Understood - for the terms it would already be enormously helpful, can appreciate this is a very tricky request / with significant implications

[RamanDamayeu]

As discussed at #1680 (comment) we're going to implement in release 0.27.0 only part for term, in particular:

"update_at"property in items with "/api/terms" responses
update_at_range_start_date_time and update_at_range_end_date_time input parameters to filter out responses based on update_at property of terms
The part for data entities needs much more redesign and effort so moved from the scope as of now.