ices-tools-dev / ices_dataportal

This repository will support the development of the new ICES Data Portal.

ICES DATA PORTAL REVAMP

Since the development of the existing ICES data portal, the portfolio of data products at ICES has grown and diversified, technology has brought new opportunities, and the policy landscape has changed. It is therefore time to rethink the user story for the ICES Data Portal, which this short document aims to do.

This repository will support the development of the next generation of http://ecosystemdata.ices.dk/

Objectives

The ICES Data portal should:

a) Be an online searchable catalogue of data managed by ICES

b) Present summaries of these data in a map viewer

c) Allow downloading of data in known international formats

d) Enable downloading detail data in native formats (i.e. DATRAS, Dome views, etc)

Specific Requirements

Content discovery

• A catalogue of the data managed at ICES.
• Accommodate indicators/data products from existing ICES systems (SAG, DATRAS Indices, bycatch risk, etc)

Viewing

Accommodate Reference layers (internal and external sourced). Being able to view the layers (base data, indicator, reference layer) in the map. Be able to query the data geospatially (eg using the map to draw an area) Having a user area to:

• Access restricted layers (eg. specific to a working group).
• Save queries (to view later).
• Upload user layers to view with the ICES layers.

Downloading

Being able to download the data in the native format or in an international format.

Current draft of the workflow:

ices_dataportal's Issues

Issues with E&L

1. no tooltip for the E&L is implemented
2. Downloaded files are all named the same. they should be more intuitive, and not named same

Design DataSet Microservice template

Need to define interface for all services to get data and metadata for each dataset

Topics: Fishing and the Selective extraction of species are 1:1

Investigate if both topics are needed, or they will always be 1:1

Citation and Disclaimer breaking individual citation and disclaimers

I just noticed that the data portal seem to overrule the individual data portals disclaimer which self shouldn’t happened or be allowed. Doing this you break the individual data portals disclaimer yourself.

Instead you could / should tell the downloader to read the individual disclaimers.

Hjalte

From: [email protected] [email protected]
Sent: 10 May 2021 17:59
To: Mehdi Abbasi [email protected]
Cc: Accessions [email protected]; Accessions [email protected]
Subject: ICES Data Portal - Download file

The data you have requested to download is now available
https://data.ices.dk/downloads?guid=3ffb4c6c-c856-4e2d-bb63-76f4b182a034

This link will be available for 7 days.
DATA CITATION
Data sources should be acknowledged by a citation. Detailed information and examples of citations can be found in ICES Data Policy: https://ices.dk/data/guidelines-and-policy/Pages/ICES-data-policy.aspx.

A minimum requirement citation would include the following:
“ICES Data Portal, 2021.ICES, Copenhagen.”

Extended citation:
ICES Data Portal, Extraction 10 May 2021 of data from https://data.ices.dk. ICES, Copenhagen

Please acknowledge the following data sources when using the data:
Acoustic trawl surveys: Biotic data in acoustic trawl surveys. 2021. ICES 2021
Acoustic trawl surveys: Acoustic data in acoustic trawl surveys. 2021. ICES 2021
Fish trawl surveys: Biotic data in fish trawl surveys. 2021. ICES 2021

For any questions or comments, please contact [email protected]
Kind regards,
ICES Data Centre

International Council for the Exploration of the Sea
Email: [email protected]
Web: https://www.ices.dk

Tel: [phone] Fax: [phone]
H.C.Andersens Boulevard 44 - 46
1553 Copenhagen V.Denmark

Read the latest ICES Data Centre e - News https://www.ices.dk/data/Pages/default.aspx

---

This email has been generated by ICES Data Portal

Search edit: not intuitive

It is hard for the first-time users to identify how to get back to their layer selection.
Maybe having a word 'Edit' would help?

Alternative map icon to re-do the data search?

The current icons could be improved, as the magnifying glass can be mistaken for zooming in and out

something like this could help?

Or like this?

Link to the metadata

Ensure that metadata, and the portal downloads have relevant link(s) back to the resources for the specific data type i.e. manuals, handbooks etc.

addition of selection criteria in Dataset themes tab

The ICES data portal (http://data.ices.dk/view-map) serves as a good starting point for a data search. It is however recommended that on the ‘Dataset themes’ tab a (number of) year(s) can be selected. In this manner everyone can easily create an overview of different survey types in a specific year.

(from the WKPILOT 2022 report)

Draw a polygon to filter data

The user will be able to draw a polygon and then get a filtered dataset based on the polygon;

Proposal for slightly amended layout

To me, a slight change in standard lay-out would make the map more intuitive: all buttons directly related to 'the things you see' (layers) on one side; i.e. top left and 'the explanation of what you see and actions for further use' (i.e. download option, Legend and Select criteria) on one side; my preference would be the right side as there is a lot of land on that side of the map. See png for example, and please discuss with other users what their preference is/would be.

Map - profiler is not intuitive

How to get the profiler is not quite clear for users.
How can we improve?

download of file after selecting single species

I like the appearance of the portal. I played around and will report findings on a case by case basis.

What I did:
Dataset Themes: Fish trawl surveys
Advanced search:
Time period: 2018-2021
Data Criteria:
Survey=Sole Net Survey and
Species=Cancer pagurus
Download data

Expectation 1: I see the locations where Cancer pagurus has been caught in the survey in those years
Preference 1: I see (more or less) how many specimens of the species were caught, including the locations where it wasn't caught.
Expectation 2: the download contains the (a) haul information for all hauls (irrespective if the species has been caught or not), (b) the catch information for the selected species.

Expectation 1 is met, Preference 1 not.
Expectation 2 is partly met, for HH. But for HL I receive the catch information for all species in the selected years in the survey. One of the added values of this portal would be that I can easily select data for a single (or a subset) of species over multiple surveys/data sources.

Conditions for data criteria

Users expect to have conditions as 'or' for the same reference list, and 'and' for the different lists

Proposal for limitation data profile

following up on DIG discussion on 20 May, a proposal for a less extensive data profile (which may be a difficult term in the light of the DIG 'Data profiling' flow diagram). This could then maybe be presented as a drop-down option in the legend, as currently is done in the 'Search criteria' box.

Please check with others.

Linkage to DOI’s

Linkage to DOI’s; accessions implementation

Development of WebServices

Implied but we should be specific about building on, and using web services as the basis of the whole portal

User management

need to define user authentication method, ActiveDirectory or independent users?

Map view - SAG

Area-based view was confusing for users.
Can we come-up with an alternative?

Incorrect link from meta data catalogue to eggs and larva portal

When downloading data for the Eggs and larvae database there is a link that goes to the Oops -page, from the meta data catalogue (https://gis.ices.dk/geonetwork/srv/eng/catalog.search#/metadata/eea6d365-6d6b-4b24-9c92-e75529500d56).
The link is:
http://www.ices.dk/marine-data/data-portals/Pages/Eggs-and-larvae.aspx

Topics: checkboxes are not intuitive

Checkboxes in the Topics are not self-explanatory, and the big icons are the first ones to be pressed.
Maybe we could play with sizes? Add text?

Add chart summary

Add chart summary to front end portal functionality

• task 1
• task 2

Sag to be included

SAG will be included, yet to be defined how this will be filtered by area selection (intersect or cookie cutter)

Feedback from Luca

I think the new interface is really intuitive and well built, congratulations!
On the main page, where each database is represented by a rectangle of different colour and sizes, I think the tick box on the upper left could be larger. In addition, it would be great (if possible) to be able to click on the entire rectangle to select and/or see sub-datasets instead of on the icon in the lower right corner.

Another observation:
Yesterday I tried to look at the acoustic trawl surveys without subsetting for any year. As expected, the map took a long time to load and every change in zoom cause a temporary freeze. I also tried to download the data, but when I opened the zip folder I had only a disclaimer and an empty zip folder. However, I have to add that I tried again today during the workshop to look at the same dataset, but I subset it using a polygon and a short 2 years period and both the visualisation and the subsequent download were successful.
For now, it is all I have but I will keep experimenting to provide further feedback.

MAP - profiler info

Information included to the profiler is too detailed, which adds up in the amount of profilers per point.
Each dataset to revise the needed minimum information in the profiler. (ie for DATRAS: Survey, country, ship)

date-time format in the data profiler

The date-time format in the data profiler is the wrong ex. localid -> Ocean_25730176

Topics: adjust the size of icons and text in drill-down

In f.ex. Fish or Research, current view in the drill-down cuts off the pictograms and text

data criteria: filtering

users expect certain filtering of data criteria, not only by a dataset, but also by other criteria, i.e. only list of Danish ships for country DK

Download: how to understand the download format

We do not show any straightforward reference to the download formats.
Users would appreciate a hint on that. disclaimer is too vague for the purpose.
dedicated PDF or a crystal-clear reference to a format would be better