ISWC 2021 Call For Challenges
A great way to advance the state of the art in a given domain is to create competition. We invite you to propose an ISWC 2021 Challenge, in which you define an open competition on a problem of your choice within the Semantic Web domain.
Proposal Stage
ISWC 2021 will run multiple challenges with the aim to evaluate and compare software solutions for the Semantic Web in a systematic way. In order to propose a challenge, we ask you to:
- ● specify a task to be addressed
- ● provide an evaluation dataset
- ● define evaluation measures to compare the performance of participating systems
Challenge Stage
After acceptance of the challenge to ISWC 2021, challenge organizers are responsible for:
- ● providing a website
- ● publishing challenge tasks, datasets, and measures
- ● selecting participants who can present their solutions at ISWC 2021
Showcase Stage
At ISWC 2021, each challenge will receive one slot during the main conference, during which contestants present their accepted solutions. Participants will also have the option to present their systems during the poster and demo session.
Topics
For ISWC 2021, Challenge proposals are invited for any challenge involving Semantic Web tasks, including but not limited to
- ● Ontology and knowledge graph alignment
- ● Ontology and knowledge graph quality assurance (QA)
- ● Knowledge graph construction and refinement
- ● Query and reasoning scalability
- ● Open information extraction
- ● Semantic Web and data mining
- ● Link prediction
- ● Question answering
- ● Stream processing and reasoning
- ● Semantic table understanding
ISWC 2021 will be a virtual conference that will take place on 24-28 October. Recent Challenge examples can be found at https://iswc2020.semanticweb.org/program/semantic-web-challenges/ and
https://iswc2019.semanticweb.org/challenges/
Proposal Submission Guidelines
The contents of a proposal should include the following (in 2–4 pages)
- ● names and affiliations of the organizers
- ● description of the challenge, including its relevance to the Semantic Web community
- ● process for selecting, acquiring, and preparing the data for training and evaluation, including conditions of availability
- ● procedure for evaluating the performance of participating systems, including the availability of evaluation software
- ● a timeline based on the template for all Challenges provided below
- ● number of expected participants with supporting evidence (expression of interests, discussions in forum/mailing lists, attendance in previous editions, etc.)
- ● any other relevant information, including previous evaluations for the proposed challenge
Selection criteria
- ● Relevance to the Semantic Web community
- ● Potential number of interested participants
- ● Rigor and transparency of task description and evaluation procedure
Please submit through EasyChair and select the track “Challenge-Proposal”.
Important Dates
Submission Deadline for Challenge Proposals: March 15, 2021
Notification of Challenge Acceptance : April 5, 2021
The tentative timeline template for organizing a Challenge is as follows: All deadlines are 23:59 AoE (anywhere on earth).
