Tornike Tsereteli

I am a Machine Learning (ML) Researcher and Engineer specializing in Natural Language Processing (NLP). Driven by a passion to solve real-world problems, I am currently training Large Language Models (LLMs) to assist researchers in understanding and reasoning about scientific texts. I also enjoy building high-quality data pipelines (garbage in, garbage out) and developing prototypes (fail fast principle). I am currently on the job market. Feel free to reach out about relevant ML Engineering roles.

latest posts

Jul 13, 2022	Survey Variable Identification
Oct 28, 2020	Rick and Morty story generation with GPT2 using Transformers and Streamlit in 57 lines of code

selected publications

Under Review
Enriching Social Science Research via Survey Item Linking

Tornike Tsereteli, Daniel Ruffinelli, and Simone Paolo Ponzetto

2024

Abs Bib PDF Code

Questions within surveys, called survey items, are used in the social sciences to study latent concepts, such as the factors influencing life satisfaction. Instead of using explicit citations, researchers paraphrase the content of the survey items they use in-text. However, this makes it challenging to find survey items of interest when comparing related work. Automatically parsing and linking these implicit mentions to survey items in a knowledge base can provide more fine-grained references. We model this task, called Survey Item Linking (SIL), in two stages: mention detection and entity disambiguation. Due to an imprecise definition of the task, existing datasets used for evaluating the performance for SIL are too small and of low-quality. We argue that latent concepts and survey item mentions should be differentiated. To this end, we create a high-quality and richly annotated dataset consisting of 20,454 English and German sentences. By benchmarking deep learning systems for each of the two stages independently and sequentially, we demonstrate that the task is feasible, but observe that errors propagate from the first stage, leading to a lower overall task performance. Moreover, mentions that require the context of multiple sentences are more challenging to identify for models in the first stage. Modeling the entire context of a document and combining the two stages into an end-to-end system could mitigate these problems in future work, and errors could additionally be reduced by collecting more diverse data and by improving the quality of the knowledge base.
@misc{tsereteli2024enrichingsocialscienceresearch, title = {Enriching Social Science Research via Survey Item Linking}, author = {Tsereteli, Tornike and Ruffinelli, Daniel and Ponzetto, Simone Paolo}, year = {2024}, eprint = {2412.15831}, archiveprefix = {arXiv}, primaryclass = {cs.DL}, url = {https://arxiv.org/abs/2412.15831}, }

ECIR

VADIS – A Variable Detection, Interlinking and Summarization System

Yavuz Selim Kartal, Muhammad Ahsan Shahid, Sotaro Takeshita, and 4 more authors

In Advances in Information Retrieval. A live demo can be found here , 2024

Abs DOI Bib Code

The VADIS system addresses the demand of providing enhanced information access in the domain of the social sciences. This is achieved by allowing users to search and use survey variables in context of their underlying research data and scholarly publications which have been interlinked with each other.

@inproceedings{10.1007/978-3-031-56069-9_22,
  author = {Kartal, Yavuz Selim and Shahid, Muhammad Ahsan and Takeshita, Sotaro and Tsereteli, Tornike and Zielinski, Andrea and Zapilko, Benjamin and Mayr, Philipp},
  editor = {Goharian, Nazli and Tonellotto, Nicola and He, Yulan and Lipani, Aldo and McDonald, Graham and Macdonald, Craig and Ounis, Iadh},
  title = {VADIS -- A Variable Detection, Interlinking and Summarization System},
  booktitle = {Advances in Information Retrieval},
  year = {2024},
  publisher = {Springer Nature Switzerland},
  address = {Cham},
  pages = {223--228},
  isbn = {978-3-031-56069-9},
  doi = {https://doi.org/10.1007/978-3-031-56069-9_22},
}

SDP

Overview of the SV-Ident 2022 Shared Task on Survey Variable Identification in Social Science Publications

Tornike Tsereteli, Yavuz Selim Kartal, Simone Paolo Ponzetto, and 3 more authors

In Proceedings of the Third Workshop on Scholarly Document Processing, Oct 2022

Abs Bib PDF Code

In this paper, we provide an overview of the SV-Ident shared task as part of the 3rd Workshop on Scholarly Document Processing (SDP) at COLING 2022. In the shared task, participants were provided with a sentence and a vocabulary of variables, and asked to identify which variables, if any, are mentioned in individual sentences from scholarly documents in full text. Two teams made a total of 9 submissions to the shared task leaderboard. While none of the teams improve on the baseline systems, we still draw insights from their submissions. Furthermore, we provide a detailed evaluation.

@inproceedings{tsereteli-etal-2022-overview,
  title = {Overview of the {SV}-Ident 2022 Shared Task on Survey Variable Identification in Social Science Publications},
  author = {Tsereteli, Tornike and Kartal, Yavuz Selim and Ponzetto, Simone Paolo and Zielinski, Andrea and Eckert, Kai and Mayr, Philipp},
  editor = {Cohan, Arman and Feigenblat, Guy and Freitag, Dayne and Ghosal, Tirthankar and Herrmannova, Drahomira and Knoth, Petr and Lo, Kyle and Mayr, Philipp and Shmueli-Scheuer, Michal and de Waard, Anita and Wang, Lucy Lu},
  booktitle = {Proceedings of the Third Workshop on Scholarly Document Processing},
  month = oct,
  year = {2022},
  address = {Gyeongju, Republic of Korea},
  publisher = {Association for Computational Linguistics},
  url = {https://aclanthology.org/2022.sdp-1.29},
  pages = {229--246},
}