SemTab: Semantic Web Challenge on Tabular Data to Knowledge Graph Matching

Organizers: Kavitha Srinivas, Ernesto Jimenez-Ruiz, Oktie Hassanzadeh, Jiaoyan Chen, Vasilis Efthymiou, Juan F. Sequeda, and Vincenzo Cutrona

Tabular data to Knowledge Graph matching is the process of assigning semantic tags from knowledge graphs (e.g., Wikidata or DBpedia) to the elements of a table. This task is a challenging problem for various reasons, including the lack of metadata (e.g., table and column names), the noisiness, heterogeneity, incompleteness and ambiguity in the data. The results of this task provide significant insights about potentially highly valuable tabular data, as recent works have shown, enabling a new family of data analytics and data science applications. SemTab provides a common framework to conduct a systematic evaluation of state-of-the-art systems.

Website: http://www.cs.ox.ac.uk/isg/challenges/sem-tab/

Semantic Reasoning Evaluation Challenge (SemREC)

Organizers: Gunjan Singh, Raghava Mutharaju, and Pavan Kapanipathi

Despite the development of several ontology reasoning optimizations and alternative reasoning approaches such neuro-symbolic methods, the existing methods still cannot deal with very expressive ontology languages. To find and improve these performance bottlenecks of the reasoners, we ideally need several real-world ontologies that span the broad spectrum in terms of their size and expressivity. However, that is often not the case. One of the potential reasons for the ontology developers to not build ontologies that vary in terms of size and expressivity is the performance bottleneck of the reasoners. This challenge includes three tasks that aim to deal with this chicken and egg problem.

Task-1. Submit a challenging real-world ontolog.

Task-2. Submit a description logic reasoner that makes use of traditional techniques such as tableau algorithms and completions rules.

Task-3. Submit an ontology/RDFS reasoner that makes use of neural-symbolic techniques for reasoning and optimization.

Website: https://semrec.github.io

SMART 2021: Semantic Answer Type and Relation Prediction Task

Organizers: Nandana Mihindukulasooriya, Mohnish Dubey, Alfio M Gliozzo, Jens Lehmann, Axel-Cyrille Ngonga Ngomo, and Ricardo Usbeck

SMART 2021 is the Semantic Answer Type and Relation Prediction task. Classifying a question's answer type plays a key role in Knowledge Graph Question Answering (KGQA). The answer type classification is the task of determining the type of the expected answer based on the query. In literature, such answer type classifications are performed as a short-text classification task. We propose that a more granular answer type classification is possible and valuable using popular Semantic Web ontologies such as DBpedia and Wikidata. Also, we will introduce the subtask of relation prediction as another step towards a functional KGQA system. This decision is based on last year's systems performances. A question can contain one or more relations from an ontology connecting the entities and classes directly or indirectly mentioned in the question. Participants will be able to choose to participate in one or both tasks using their systems.

Website: https://smart-task.github.io/

CANCELLED - Bio2RDF and Kibio federated query in Life Science challenge

Organizers: François Belleau, Régis Ongario-Carcy, Mickaël Leclerc, and Arnaud Droid

How can we run the Wikidata FAIR paper query without using Wikidata SPARQL endpoint (https://tinyurl.com/53ra32bp)? Participants must demonstrate how they can obtain the same results from this query by using federated techniques. The only data services authorized are: Virtuoso SPARQL endpoints from Bio2RDF and Elasticsearch index with JSON-LD documents from Kibio.science, where each needed datasource will be made available (DOID, GO, Wikipathways, etc.)

Two tasks are proposed:

1) Using federated SPARQL query that will consume Bio2RDF’s endpoints.

2) Using any programming approach that will consume Kibio’s Elasticsearch indexes.

The solutions proposed will be benchmarked for speed in a controlled environment using docker images. In the second phase of the competition, participants will have to run a small set of unknown queries. The team with the fastest solution for all the queries will win. Will REST JSON-LD APIs do better over SPARQL endpoints? We will see.

Website: http://kibio.science/iswc2021/

CANCELLED - Semantic Food Q&A Task

Organizers: Oshani Seneviratne, Ching-Hua Chen, Pablo Meyer, Mauro Dragoni, and Tome Eftimov

There is an immense amount of food-related information on the Web, including structured semantic data sources such as DBpedia and Wikidata. Recent efforts have brought together even more semantic data sources on food, such as the FoodOn, Healthy Lifestyle Support Ontology (HeLiS) ontology, and the FoodKG. Some other structured data sources, such as the FoodBase, which is annotated with the Hansard corpus, and the FoodOntoMap, have a normalized set of semantic tags from different food ontologies. Leveraging all these semantic datasets, this ISWC'21 challenge task attempts to identify the correct food item(s) given a question.

The questions can be of three types:

(1) Factoid questions (e.g., "How much carbs are in a serving of breakfast burrito?");

(2) Comparison questions (e.g., "Does Peanut Oil have more monounsaturated fat compared to Sesame Oil?''); and

(3) Constraint questions (e.g., "Suggest a food item without peanuts?").

The challenge participants will have access to various semantic knowledge sources on food and an evaluation dataset for the above types of questions with sample answers. The range of questions used in the challenge will consist of eliciting food concepts to understand the relations between food, ingredients, nutritional values, human health, and any contextual information applicable to a food recommendation.

Website: https://foodqa.github.io/

CANCELLED - Operationalizing Linked Data in Mobile Apps Using MIT App Inventor’s Punya

Organizers: Lalana Kagal, Giuseppe Loseto, Evan W. Patton, Floriano Scioscia, Oshani Seneviratne, and William Van Woensel

Semantic Web technologies can help overcome the difficulty of building interoperable, intelligent, and fully-fledged mobile apps through domain ontologies and semantic representation formalisms. Punya (http://punya.mit.edu) is a drag-and-drop visual programming environment based on MIT App Inventor (https://appinventor.mit.edu) that incorporates many semantic technologies to democratize the ability to consume, produce, and act on Linked Data. One can build apps that consume Linked Data by querying a remote SPARQL endpoint and generate Linked Data through forms annotated with a domain ontology. In this ISWC'21 challenge, participants are expected to build an intelligent mobile app using the Punya framework that connects with online Linked Data sources and services, nearby IoT devices or on-device sensors, and/or implement expert system features by producing, consuming, and processing Linked Data. To that end, participants may extend the Punya framework itself with new Linked Data components or extend existing ones. Participants should target the synergistic combination of components, providing added value in a mobile setting that goes beyond any individual component. Apps should tackle relevant and challenging real-world scenarios that impact health, the environment, and society.

The list of suggested themes includes

Linked Data for Social Good;

Better Resource Allocation (e.g., water scarcity, natural disaster response);

Climate Change;

Ending Poverty;

Health Care (e.g., COVID-19, mental health);

Learning and Working Remotely;

Living Together;

Active Aging;

Social and Racial Justice; and

Social Impact of Artificial Intelligence (e.g., biases, opportunities, equitability).

Website: http://punya.mit.edu/challenge

All the Agents Challenge (ATAC)

Organizers: Tobias Käfer, Andreas Harth, Andrei Ciortea, and Victor Charpenay

Agents on the Web have been a part of the vision for the Semantic Web. While it has been noted that the original Semantic Web vision has not yet fully materialised, recent progress seems to indicate that we may now be at a stage where: (i) developments such as the Web of Things may unlock new practical use cases for agents on the Web, and (ii) the substrate provided by recent Semantic Web technologies (e.g., Linked Data Platform, Linked Data Notifications, SoLiD PODs) may open new perspectives on the integration of Semantic Web with agent technologies. This bridging of technologies and communities has also been the topic of the Dagstuhl seminar "Autonomous Agents on the Web" held in Feburary 2021. Therefore, for the 20th edition of the ISWC conference and in spirit of Jim Hendler's op-ed that asked "Where are all the agents,?" we want to pose the All-the-Agents-Challenge (ATAC) to the Semantic Web community: The challenge to build agents that do things on Linked Data.

Website: http://purl.org/atac/2021

Semantic Web Challenges