Important Dates

* All deadlines are calculated at 11:59 pm
UTC-12 hours

Trial Data Ready Jul 15 (Fri), 2022
Training Data Ready Sep 30 (Fri), 2022
Evaluation Start Jan 10 (Tue), 2023
Evaluation End Jan 31 (Tue), 2023
System Description Paper Submission Due Feb 1 (Wed), 2023
Notification to Authors Mar 1 (Wed), 2023
Camera-ready Due Apr 1 (Sat), 2023
Workshop TBD



Complex named entities (NE), like the titles of creative works, are not simple nouns and pose challenges for NER systems (Ashwini and Choi, 2014). They can take the form of any linguistic constituent, like an imperative clause (“Dial M for Murder”), and do not look like traditional NEs (Persons, Locations, etc.). This syntactic ambiguity makes it challenging to recognize them based on context. We organized the MultiCoNER task (Malmasi et al., 2022) at SemEval-2022 to address these challenges in 11 languages, receiving a very positive community response with 34 system papers. Results confirmed the challenges of processing complex and long-tail NEs: even the largest pre-trained Transformers did not achieve top performance without external knowledge. The top systems infused transformers with knowledge bases and gazetteers. However, such solutions are brittle against out of knowledge-base entities and noisy scenarios like the presence of spelling mistakes and typos. We propose MultiCoNER II which represents novel challenges through new tasks that emphasize the shortcomings of the current top models.

MultiCoNER II features complex NER in these languages:

  1. English
  2. Spanish
  3. Hindi
  4. Bangla
  5. Chinese
  6. Swedish
  7. Farsi
  8. French
  9. Italian
  10. Portugese
  11. Ukranian
  12. German

The following figure shows the fine-grained taxonomy of the dataset. Taxonomy

MultiCoNER I: SemEval 2022 Task 11

References

Communication

Anti-Harassment Policy

SemEval highly values the open exchange of ideas, freedom of thought and expression, and respectful scientific debate. We support and uphold the NAACL Anti-Harassment policy. Participants are encouraged to send any concerns or questions to the NAACL Board members, Priscilla Rasmussen and/or the workshop organizers.