svg image

NTCIR History

NTCIR-Lifelog is a core task of the NTCIR-17 conference. This core task aims to advance the state-of-the-art research in lifelogging as an application of information retrieval. The methodology employed for the lifelog task at NTCIR-17 is based on the successfully deployed methodology from NTCIR-12-16.

More Papers

Tasks

Lifelog Semantic Access SubTask - LSAT

For this year, we still focus on the most popular task from the previous lifelog tasks. The Lifelog Semantic Access Task (LSAT) is a known-item search task that can be undertaken in an interactive or automatic manner. In this sub task, the participants have to retrieve a number of specific moments in a lifelogger's life. We define moments as semantic events, or activities that happened throughout the day. The task can best be compared to a known-item search task.

Example search tasks include:
  • The moment(s) where I was boarding an A380 on my way to China
  • Find the moment(s) where I am playing with my phone during a dinner date.
  • Find the moment(s) where I am preparing breakfast before heading to the airport to return home.

The LSAT sub task can be undertaken by participants in either an automatic manner or an interactive manner. Automatic runs assume that there was no user involvement in the search process beyond specifying the query. Interactive runs assume that there is a user involved in the search process that generates a query and selects which moments are considered correct for each topic. It is important that the correct task submission type is identified by participants when submitting runs. There will be 41 topics prepared for the task, which will be available from the NTCIR server.

Lifelog Insight SubTask - LIT

Lifelog Insight subTask is to explore knowledge mining and visualisation of lifelogs by setting general challenges for the participants to address.

The aim of this subtask is to gain insights into the lifelogger's diet and health. It follows the idea of the Quantified Self movement that focuses on the visualization of knowledge mined from self-tracking data to provide "self-knowledge through numbers". Participants are requested to generate new types of visualisations and insights about the life of the lifeloggers by generating a themed diary or themed insights related to the diet and health of the lifeloggers. The submissions are not evaluated in the traditional sense, but will form the basis of an interactive session at NTCIR-17.
Lifelog Question Answer SubTask - LQAT

Lifelog Question Answer subTask is to encourage comparative progress on the important Q&A topic from lifelogs. For this subtask, an augmented 85-day lifelog collection with over 15,000 multiple-choice questions and baseline will be provided, participants can train and compare their lifelog QA models.

Dataset

NTCIR17-Lifelog reuses an existing dataset, the LSC'23 dataset, which is a multimodal dataset that is four months in size, from one active lifelogger. The dataset consists of three files, each of which is password protected and details of how to access these datasets can be sourced from liting.zhou@dcu.ie with the competed agreement forms as descried below. The dataset comprises of:

  • Core Image Dataset of wearable camera images, fully redacted and anonymised in 1024 x 768 resolution, captured using a Narrative Clip device. These images were collected during 2019-2020. All faces and readable text have been removed, as well as certain scenes and activities manually filtered out to respect local privacy requirements.
  • Metadata for the collection, consisting of textual metadata representing time and locations, etc…
  • Visual Concepts extracted from the non-redacted version of the visual dataset.
  • Additional Data: The team of MyScéal has also provided a supplementary metadata file (vaisl_gps.csv) containing semantic names for location as well as some improvements to the original raw location data. The content and process of creating this file is described in the paper VAISL: Visual-Aware Identification of Semantic Locations in Lifelog. The Voxento developer (Ahmed) has made available a custom location metadata that solves an irregularity with flights (locations on flightpath being recorded); the metadata stores flight locations as departing airport - arrival airport. It is shared here for anyone who could find it useful.
For access to the full dataset, please email liting.zhou@dcu.ie with the competed agreement forms as descried below. Please note that participants are also expected to register on the NTCIR-17 Website in order to participate.

NTCIR-Lifelog's participants are required to sign two forms to access the datasets, an organisational agreement form for your organisation (signed by the research team leader) and an individual agreement form for each member of the research team that will access the data. The organisation agreement form should be sent to the lifelog task organisers (liting.zhou@dcu.ie) in PDF format. The individual agreement form must be signed by all researchers who will use the data and kept by the organisation on file. It should not be sent to the organisers, unless requested at a later date.

Submission

Each participating group should contact the organisers to get the collection used in the task. The required forms must be filled in as per the dataset instructions. When the dataset has been downloaded, then the participating team can index the content in whichever form is desired. There is no limitation on the type of data processing, enrichment or indexing process to be employed. Some participants may choose to index the provided metadata into a conventional inverted index, while others may choose to enhance the provided metadata using automated or semi-automated means, then index the data according to their preference.

Then the participant must decide between two types of run to submit. Each run must be either automatic or interactive and must be appropriately labeled. The unit of retrieval (ranking) is the image ID (without JPG file extension).

  • Automatic runs assume that there was no user involvement in the search process beyond specifying the initial query, which can only happen once for each topic. The search system generates a ranked list of up to 100 images for each topic. There is no time limit on how long it can take for an automatic run. We assume that any human involvement in generating a query from the topic is a once-off process that is not iterative and dependent on the results of a previous execution of a query for that topic (i.e. no human-influenced relevance feedback mechanism can be applied to an automatic run). The submission file format includes SCORE to capture the score of every image as returned by the ranking algorithm. In the automatic run case the SECONDS-ELAPSED column should always have a value equal to 0, since it is only relevant for Interactive runs.
  • Interactive runs assume that there is a user involved in the search process that generates a query and selects which moments are considered correct for each topic. This may be a single phase, or may contain multiple phases of relevance feedback or query reformulation. In interactive runs, the maximum time allowed for any topic should be 300 seconds. The submission file format includes SECONDS-ELAPSED to capture the time taken to find every moment. In the interactive runs, mo more than 100 images may be submitted for any query also. In the interactive run, the SCORE value should be equal to 1. For interactive runs, the seconds elapsed should be equal to the number of seconds (from zero) that it took the user to find a particular item in the submission. For example, if a user in an interactive run found one potentially relevant item at 5 seconds, another at 15 seconds and a third at 255 seconds, then there would be three lines in the CSV file for that run, each of which has a different value for the SECONDS-ELAPSED column. It is important to accurately record this value since it will be used to calculate run performance at different time cutoffs (e.g. 10 seconds, 60 seconds, etc...).


LSAT Submission Format
Each participating group should contact the organisers to get the collection used in the task. The required forms must be filled in as per the dataset instructions. When the dataset has been downloaded, then the participating team can index the content in whichever form is desired. There is no limitation on the type of data processing, enrichment or indexing process to be employed. Some participants may choose to index the provided metadata into a conventional inverted index, while others may choose to enhance the provided metadata using automated or semi-automated means, then index the data according to their preference.

For every topic, every image considered relevant should have one line in the CSV file. For some topics there will be only one relevant item (one line in the submission), for others there will be many relevant items (many lines in the submission), up to 100. It is also possible that no relevant items are found for a topic, so then there should be no entry in the file for the topic.

The format of the CSV file for an automatic run would be as follows: GROUP-ID, RUN-ID, TOPIC-ID, IMAGE-ID, SECONDS-ELAPSED, SCORE
...
DCU, DCULSAT01, 16001, u1_2016-08-15_112559, 0, 1.0
DCU, DCULSAT01, 16001, u1_2016-08-15_120354, 0, 1.0
...
The format of the CSV file for an interactive run would be as follows:GROUP-ID, RUN-ID, TOPIC-ID, IMAGE-ID, SECONDS-ELAPSED, SCORE
...
DCU, DCULSAT01, 16001, u1_2016-08-15_112559, 33, 1.0
DCU, DCULSAT01, 16001, u1_2016-08-15_120354, 54, 1.0
DCU, DCULSAT01, 16001, u1_2016-08-15_120412, 243, 1.0
...

In total there are 41 topics for this lifelog LSAT task. They are available now for download. There are two types of topics, adhoc and knownitem:
  • ADHOC - topics that may have many moments in the collection that are relevant. These topics are all new.
  • KNOWNITEM - topics with one (or few) relevant moments in the collection.
The format of the topic is as follows:
  • ID - A unique identifier of every topic
  • Type - identifying each topic as being either adhoc, or knownitem.
  • UID - a user identifier. Always u1 (user 1) for this collection.
  • Title - a title of the query used for identification
  • Description - a descriptive query that would represent the information need of the user
  • Narrative - additional details about the information need that helps to define what is correct and incorrect.

Therefore a sample query would be as follows:

<topic>
<id>16000</id>
<type>knownitem</type>
<uid>u1</uid>
<title>Eating fruit</title>
<description>Find the time when I was eating both mandarins and apples at my work desk.</description>
<narrative>The lifelogger was at work and eating mandarin oranges and apples at his desk. Any moment showing such activities are considered relevant. Eating both fruits in a location other than the lifelogger's desk is not considered to be relevant. </narrative>
</topic>

LIT Submission Format
It is not necessary to submit your insights task output by the due date. We do, however, request that you send us a short abstract of your LIT subtask visualisations and insights to (liting.zhoug@computing.dcu.ie) by the due date. This abstract should be about one paragraph (10 lines) in length and may include the visualisation if ready. It is expected that the submission will be described in the participating group’s NTCIR-17 paper and will be demonstrated at the NTCIR-17 conference.
LQAT Submission Format
You can downlaod the QA pairs here. The format of LQAT is CSV file with question id and answer in index. For example:

GROUP-ID, QUESTION-ID, ANSWER-IDX
...
DCU, test_mc_0, 4
DCU, test_mc_1, 0
...