Skip to main content
Springer Nature Link
Log in

CLEF eHealth 2019 Evaluation Lab

  • Conference paper
  • First Online:
Advances in Information Retrieval (ECIR 2019)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11438))

Included in the following conference series:

Abstract

Since 2012 CLEF eHealth has focused on evaluation resource building efforts around the easing and support of patients, their next-of-kins, clinical staff, and health scientists in understanding, accessing, and authoring eHealth information in a multilingual setting. This year’s lab offers three tasks: Task 1 on multilingual information extraction; Task 2 on technology assisted reviews in empirical medicine; and Task 3 on consumer health search in mono- and multilingual settings. Herein, we describe the CLEF eHealth evaluation series to-date and then present the 2019 tasks, evaluation methodology, and resources.

LK, LG & HS co-chair the CLEF eHealth lab and contributed equally to this paper. MN, EK & RS & LA & DL, and J & JP & GZ lead 2019 lab Tasks 1–3, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
eBook
USD 39.99
Price excludes VAT (USA)
Softcover Book
USD 54.99
Price excludes VAT (USA)

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://sites.google.com/site/clefehealth/ (last accessed on 19 October 2018).

  2. 2.

    https://www.animaltestinfo.de/ (last accessed on 18 October 2018).

  3. 3.

    http://commoncrawl.org/ (last accessed on 19 October 2018).

  4. 4.

    https://sites.google.com/site/clefehealth/datasets (last accessed on 18 October 2018).

References

  1. Bert, B., et al.: Rethinking 3R strategies: digging deeper into AnimalTestInfo promotes transparency in in vivo biomedical research. PLoS Biol. 15(12), 1–20 (2017). https://doi.org/10.1371/journal.pbio.2003217

    Article  Google Scholar 

  2. Goeuriot, L., et al.: ShARe/CLEF eHealth evaluation lab 2013, task 3: information retrieval to address patients’ questions when reading clinical reports. CLEF 2013 Online Working Notes 8138 (2013)

    Google Scholar 

  3. Goeuriot, L., et al.: An analysis of evaluation campaigns in ad-hoc medical information retrieval: CLEF eHealth 2013 and 2014. Springer Inf. Retr. J. 21(6), 507–540 (2018)

    Article  Google Scholar 

  4. Goeuriot, L., et al.: ShARe/CLEF eHealth evaluation lab 2014, task 3: user-centred health information retrieval. In: CLEF 2014 Evaluation Labs and Workshop: Online Working Notes, Sheffield, UK (2014)

    Google Scholar 

  5. Goeuriot, L., et al.: Overview of the CLEF eHealth evaluation lab 2015. In: Mothe, J., et al. (eds.) CLEF 2015. LNCS, vol. 9283, pp. 429–443. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24027-5_44

    Chapter  Google Scholar 

  6. Goeuriot, L., et al.: CLEF 2017 eHealth evaluation lab overview. In: Jones, G.J.F., et al. (eds.) CLEF 2017. LNCS, vol. 10456, pp. 291–303. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-65813-1_26

    Chapter  Google Scholar 

  7. Jimmy, Zuccon, G., Palotti, J., Goeuriot, L., Kelly, L.: Overview of the CLEF 2018 consumer health search task. In: Working Notes of Conference and Labs of the Evaluation (CLEF) Forum. CEUR Workshop Proceedings (2018)

    Google Scholar 

  8. Kanoulas, E., Li, D., Azzopardi, L., Spijker, R.: CLEF 2017 technologically assisted reviews in empirical medicine overview. In: Working Notes of Conference and Labs of the Evaluation (CLEF) Forum. CEUR Workshop Proceedings (2017)

    Google Scholar 

  9. Kanoulas, E., Li, D., Azzopardi, L., Spijker, R.: CLEF 2018 technologically assisted reviews in empirical medicine overview. In: Working Notes of Conference and Labs of the Evaluation (CLEF) Forum. CEUR Workshop Proceedings (2018)

    Google Scholar 

  10. Kelly, L., Goeuriot, L., Suominen, H., Névéol, A., Palotti, J., Zuccon, G.: Overview of the CLEF eHealth evaluation lab 2016. In: Fuhr, N., et al. (eds.) CLEF 2016. LNCS, vol. 9822, pp. 255–266. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-44564-9_24

    Chapter  Google Scholar 

  11. Kelly, L., et al.: Overview of the ShARe/CLEF eHealth evaluation lab 2014. In: Kanoulas, E., et al. (eds.) CLEF 2014. LNCS, vol. 8685, pp. 172–191. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11382-1_17

    Chapter  Google Scholar 

  12. Névéol, A., et al.: Clinical information extraction at the CLEF eHealth evaluation lab 2016. In: Balog, K., Cappellato, L., Ferro, N., Macdonald, C. (eds.) CLEF 2016 Working Notes. CEUR Workshop Proceedings (CEUR-WS.org) (2016). http://ceur-ws.org/Vol-1609/. ISSN 1613-0073

  13. Névéol, A., et al.: CLEF eHealth 2017 multilingual information extraction task overview: ICD10 coding of death certificates in English and French. In: CLEF 2017 Online Working Notes. CEUR-WS (2017)

    Google Scholar 

  14. Névéol, A., et al.: CLEF eHealth 2018 multilingual information extraction task overview: ICD10 coding of death certificates in French, Hungarian and Italian. In: CLEF 2018 Online Working Notes. CEUR-WS (2018)

    Google Scholar 

  15. Palotti, J., et al.: CLEF eHealth evaluation lab 2015, task 2: retrieving information about medical symptoms. In: CLEF 2015 Online Working Notes. CEUR-WS (2015)

    Google Scholar 

  16. Palotti, J., et al.: CLEF 2017 task overview: the IR task at the eHealth evaluation lab. In: Working Notes of Conference and Labs of the Evaluation (CLEF) Forum. CEUR Workshop Proceedings (2017)

    Google Scholar 

  17. Suominen, H.: CLEF 2012 Working Notes. CEUR Workshop Proceedings (CEUR-WS.org). Ed. by Forner, P., Karlgren, J., Womser-Hacker, C., Ferro, N. (2012). http://ceur-ws.org/Vol-1178/. ISSN 1613-0073

  18. Suominen, H., Kelly, L., Goeuriot, L.: Scholarly influence of the conference and labs of the evaluation forum ehealth initiative: review and bibliometric study of the 2012 to 2017 outcomes. JMIR Res. Protoc. 7(7), e10961 (2018). https://doi.org/10.2196/10961

    Article  Google Scholar 

  19. Suominen, H., et al.: Overview of the CLEF ehealth evaluation lab 2018. In: Bellot, P., et al. (eds.) CLEF 2018. LNCS, vol. 11018, pp. 286–301. Springer, Heidelberg (2018). https://doi.org/10.1007/978-3-319-98932-7_26

    Chapter  Google Scholar 

  20. Suominen, H., et al.: Overview of the ShARe/CLEF eHealth Evaluation Lab 2013. In: Forner, P., Müller, H., Paredes, R., Rosso, P., Stein, B. (eds.) CLEF 2013. LNCS, vol. 8138, pp. 212–231. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40802-1_24

    Chapter  Google Scholar 

  21. Zuccon, G., et al.: The IR task at the CLEF eHealth evaluation lab 2016: user-centred health information retrieval. In: CLEF 2016 Evaluation Labs and Workshop: Online Working Notes, CEUR-WS, September 2016

    Google Scholar 

Download references

Acknowledgements

We gratefully acknowledge the people involved in the CLEF eHealth labs as participants or organizers. We also acknowledge the many organizations that have supported CLEF eHealth labs since 2012. The CLEF eHealth 2019 evaluation lab is supported in part by the CLEF Initiative, Data61/CSIRO, a Google Faculty Research Award, and ARC DECRA grant DE180101579.

Author information

Authors and Affiliations

  1. Maynooth University, Maynooth, Ireland

    Liadh Kelly

  2. Univ. Grenoble Alpes, Grenoble, France

    Lorraine Goeuriot

  3. Research School of Computer Science, The Australian National University (ANU), Canberra, ACT, Australia

    Hanna Suominen

  4. Data61, Commonwealth Scientific and Industrial Research Organisation (CSIRO), Canberra, ACT, Australia

    Hanna Suominen

  5. Faculty of Science and Technology, University of Canberra, Canberra, ACT, Australia

    Hanna Suominen

  6. Department of Future Technologies, University of Turku, Turku, Finland

    Hanna Suominen

  7. German Federal Institute for Risk Assessment (BfR), Berlin, Germany

    Mariana Neves

  8. Informatics Institute, University of Amsterdam, Amsterdam, Netherlands

    Evangelos Kanoulas & Dan Li

  9. Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, UMC Utrecht, Utrecht University, Utrecht, Netherlands

    Rene Spijker

  10. Medical Library, Amsterdam Public Health, Amsterdam UMC, University of Amsterdam, Amsterdam, Netherlands

    Rene Spijker

  11. Computer and Information Sciences, University of Strathclyde, Glasgow, UK

    Leif Azzopardi

  12. The University of Queensland, St Lucia, QLD, Australia

    Jimmy & Guido Zuccon

  13. Vienna University of Technology, Vienna, Austria

    João Palotti

  14. Qatar Computing Research Institute, Doha, Qatar

    João Palotti

Authors
  1. Liadh Kelly
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lorraine Goeuriot
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hanna Suominen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mariana Neves
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Evangelos Kanoulas
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Rene Spijker
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Leif Azzopardi
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Dan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jimmy
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. João Palotti
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Guido Zuccon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liadh Kelly .

Editor information

Editors and Affiliations

  1. University of Strathclyde, Glasgow, UK

    Leif Azzopardi

  2. Bauhaus Universität Weimar, Weimar, Germany

    Benno Stein

  3. Universität Duisburg-Essen, Duisburg, Germany

    Norbert Fuhr

  4. GESIS - Leibniz Institute for the Social Sciences, Cologne, Germany

    Philipp Mayr

  5. Delft University of Technology, Delft, The Netherlands

    Claudia Hauff

  6. University of Twente, Enschede, The Netherlands

    Djoerd Hiemstra

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kelly, L. et al. (2019). CLEF eHealth 2019 Evaluation Lab. In: Azzopardi, L., Stein, B., Fuhr, N., Mayr, P., Hauff, C., Hiemstra, D. (eds) Advances in Information Retrieval. ECIR 2019. Lecture Notes in Computer Science(), vol 11438. Springer, Cham. https://doi.org/10.1007/978-3-030-15719-7_36

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-15719-7_36

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-15718-0

  • Online ISBN: 978-3-030-15719-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation