Call for semantic web challenge proposals



ISWC 2021 Call For Challenges


A great way to advance the state of the art in a given domain is to create competition. We invite you to propose an ISWC 2021 Challenge, in which you define an open competition on a problem of your choice within the Semantic Web domain.


Proposal Stage


ISWC 2021 will run multiple challenges with the aim to evaluate and compare software solutions for the Semantic Web in a systematic way. In order to propose a challenge, we ask you to:

  • ● specify a task to be addressed
  • ● provide an evaluation dataset
  • ● define evaluation measures to compare the performance of participating systems​

Challenge Stage


After acceptance of the challenge to ISWC 2021, challenge organizers are responsible for:

  • ● providing a website
  • ● publishing challenge tasks, datasets, and measures
  • ● selecting participants who can present their solutions at ISWC 2021

Showcase Stage


At ISWC 2021, each challenge will receive one slot during the main conference, during which contestants present their accepted solutions. Participants will also have the option to present their systems during the poster and demo session.


Topics


For ISWC 2021, Challenge proposals are invited for any challenge involving Semantic Web tasks, including but not limited to

  • ● Ontology and knowledge graph alignment
  • ● Ontology and knowledge graph quality assurance (QA)
  • ● Knowledge graph construction and refinement
  • ● Query and reasoning scalability
  • ● Open information extraction
  • ● Semantic Web and data mining
  • ● Link prediction
  • ● Question answering
  • ● Stream processing and reasoning
  • ● Semantic table understanding


ISWC 2021 will be a virtual conference that will take place on 24-28 October. Recent Challenge examples can be found at https://iswc2020.semanticweb.org/program/semantic-web-challenges/ and

https://iswc2019.semanticweb.org/challenges/


Proposal Submission Guidelines


The contents of a proposal should include the following (in 2–4 pages)

  • ● names and affiliations of the organizers
  • ● description of the challenge, including its relevance to the Semantic Web community
  • ● process for selecting, acquiring, and preparing the data for training and evaluation, including conditions of availability
  • ● procedure for evaluating the performance of participating systems, including the availability of evaluation software
  • ● a timeline based on the template for all Challenges provided below
  • ● number of expected participants with supporting evidence (expression of interests, discussions in forum/mailing lists, attendance in previous editions, etc.)
  • ● any other relevant information, including previous evaluations for the proposed challenge


Selection criteria

  • ● Relevance to the Semantic Web community
  • ● Potential number of interested participants
  • ● Rigor and transparency of task description and evaluation procedure


Please submit through EasyChair and select the track “Challenge-Proposal”.


Important Dates


Submission Deadline for Challenge Proposals: March 15, 2021

Notification of Challenge Acceptance : April 5, 2021


The tentative timeline template for organizing a Challenge is as follows: All deadlines are 23:59 AoE (anywhere on earth).



Activities Due Date
Challenge Website Ready 26 April 2021
1st Call for Challenge Participants 26 April 2021
Release of Dataset 15 May
Submission of Systems 15 July 2021
Notification of System Results 15 August 2021
Participation Invitations to Present at ISWC2021 15 August 2021
Challenge Presentations 24 - 28 October 2021


Program Chairs


Ernesto Jiménez-Ruiz

City, University of London, UK & University of Oslo, Norway

ernesto.jimenez.ruiz@gmail.com


Jiaoyan Chen

University of Oxford, UK

jiaoyan.chen@cs.ox.ac.uk


Despoina Magka

Facebook AI Research, UK

magkades@gmail.com