Youtaqa : système de questions-réponses intelligent basé sur le deep learning et la recherche d’information

CONCLUSION GÉNÉRALE

question. Après l'avoir entraîné, notre Classifieur permet désormais des performances convergentes vers l'état de l'art actuel en affichant une Accuracy égale à 87% et une Précision de 89% sur le Test Set ce qui est plus qu'acceptable en prenant compte la difficultédes tâches NLP et de la classification du texte en particulier.

Ensuite, et après avoir choisi le meilleur passage susceptible à contenir une réponse à la question posée par l'utilisateur, la tàache la plus importante était d'extraire la réponse exacte à cette question, d'oùle module d'extraction des réponses MER a étéréaliséafin d'accomplir cette tâche. Le module d'extraction des réponses a étéimplémentéen utilisant le modèle BERT et a étéentraàýnéen s'appuyant sur le DataSet SQuAD . Le module MER affiche un F1-Score égal à 92% et un Exact Match de 87% en le testant sur plus de 5 000 questions, ce score représente une contribution qui pourrait être ajoutée à la littérature.

Afin de tout concrétiser, et de fournir une expérience utilisateur digne d'un système haut niveau, nous avons implémentéune application web moderne qui satisfait les normes internationales de design.

Perspectives

Les travaux rapportés dans cette thèse ne composent qu'un simple morceau dans un puzzle de travaux supplémentaires qui doivent être réalisés pour aboutir à un système parfait. Pour cela, les perspectives envisageables afin d'améliorer ce travail sont multiples, nous citons:

-- Améliorer notre moteur de recherche en employant des méthodes basées sur le Deep Learning.

-- Implémenter un module de mise à jour automatique qui permet d'actualiser la base des documents

Wikipédia régulièrement afin d'envisager de répondre aux questions liées aux nouveaux sujets.

-- Mettre en oeuvre la version arabe et française de notre système YouTaQA pour but d'atteindre une plus

large communauté.

-- Implémenter une API afin de faciliter l'utilisation du noyau YouTaQA dans des applications tierces. Pour finir, le savoir est la seule matière qui s'accroisse quand on la partage, comme le dit Socrate, sur ce, nous avons mis à disposition notre code source sur GitHub : https://github.com/rbouadjenek/YouTaQA.

45

Bibliographie

[RNN, 2017] (2017). Basic architecture of rnn and lstm. https://pydeeplearning.weebly.com/blog/ basic-architecture-of-rnn-and-lstm. Accessed: 2020-07-25.

[VSM, 2017] (2017). information retrieval document search using vec-

tor space model in r. http://www.dataperspective.info/2017/11/

information-retrieval-document-search-using-vector-space-model-in-r.html. Ac-
cessed: 2020-07-29.

[Tra, 2018] (2018). The illustrated transformer. https://developpaper.com/

the-calculation-of-self-attention-in-bert-series/. Accessed: 2020-07-25.

[Fin, 2019] (2019). Ideas on how to fine-tune a pre-trained mo-

del in pytorch. https://medium.com/udacity-pytorch-challengers/
ideas-on-how-to-fine-tune-a-pre-trained-model-in-pytorch-184c47185a20.

[Sel, 2020] (2020). The calculation of self attention in bert series. https://developpaper.com/ the-calculation-of-self-attention-in-bert-series/. Accessed: 2020-07-25.

[Wik, 2020] (2020). Wikipedia :size comparisons. https://en.wikipedia.org/wiki/Wikipedia:Size_ comparisons. Accessed: 2020-07-25.

[Abadi et al., 2016] Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016). Tensorflow : A system for large-scale machine learning. In 12th {USENIX} symposium on operating systems design and implementation ({OSDI} 16), pages 265-283.

[Adamic et al., 2008] Adamic, L. A., Zhang, J., Bakshy, E., and Ackerman, M. S. (2008). Knowledge sharing and yahoo answers. In Proceeding of the 17th international conference on World Wide Web - WWW '08. ACM Press.

[Ahn et al., 2004] Ahn, D., Jijkoun, V., Mishne, G., M·uller, K., Rijke, M., and Schlobach, S. (2004). Using wikipedia at the trec qa track.

46

Bibliographie

[Baeza-Yates and Ribeiro-Neto, 2011] Baeza-Yates, R. A. and Ribeiro-Neto, B. A. (2011). Modern Information Retrieval - the concepts and technology behind search, Second edition.

[Bahdanau et al., 2015] Bahdanau, D., Cho, K., and Bengio, Y. (2015). Neural machine translation by jointly learning to align and translate.

[Banko et al., 2002] Banko, M., Brill, E., Dumais, S., and Lin, J. (2002). Askmsr : Question answering using the worldwide web. In Proceedings of 2002 AAAI Spring Symposium on Mining Answers from Texts and Knowledge Bases, pages 7-9.

[Baudis, 2015] Baudis, P. (2015). Yodaqa : a modular question answering system pipeline. In POSTER 201519th International Student Conference on Electrical Engineering, pages 1156-1165.

[Baudis and Sediv'y, 2015] Baudis, P. and Sediv'y, J. (2015). Modeling of the question answering task in the yo-daqa system. In Proceedings of the 6th International Conference on Experimental IR Meets Multilinguality, Multimodality, and Interaction - Volume 9283, CLEF'15, page 1156-1165, Berlin, Heidelberg. Springer-Verlag.

[Bauer and Berleant, 2012] Bauer, M. A. and Berleant, D. (2012). Usability survey of biomedical question answering systems. Human Genomics, 6(1).

[Beitzel et al., 2009] Beitzel, S. M., Jensen, E. C., and Frieder, O. (2009). MAP, pages 1691-1692. Springer US, Boston, MA.

[Benamara and Saint-Dizier, 2003] Benamara, F. and Saint-Dizier, P. (2003). Webcoop : A cooperative question-answering system on the web.

[Berant et al., 2013] Berant, J., Chou, A., Frostig, R., and Liang, P. (2013). Semantic parsing on Freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1533-1544, Seattle, Washington, USA. Association for Computational Linguistics.

[Bian et al., 2017] Bian, W., Li, S., Yang, Z., Chen, G., and Lin, Z. (2017). A compare-aggregate model with dynamic-clip attention for answer selection. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM '17, page 1987-1990, New York, NY, USA. Association for Computing Machinery.

[Blier et al., 2018] Blier, L., Wolinski, P., and Ollivier, Y. (2018). Learning with random learning rates.

[Bollacker et al., 2008] Bollacker, K., Evans, C., Paritosh, P., Sturge, T., and Taylor, J. (2008). Freebase : A collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, SIGMOD '08, page 1247-1250, New York, NY, USA. Association for Computing Machinery.

[Bordes et al., 2015] Bordes, A., Usunier, N., Chopra, S., and Weston, J. (2015). Large-scale simple question answering with memory networks. ArXiv, abs/1506.02075.

47

Bibliographie

[Bouadjenek et al., 2016] Bouadjenek, M. R., Hacid, H., and Bouzeghoub, M. (2016). Social networks and information retrieval, how are they converging? a survey, a taxonomy and an analysis of social information retrieval approaches and platforms. Information Systems, 56.

[Bouma G., 2005] Bouma G., Mur J., v. N. G. (2005). Reasoning over dependency relations.

[Brill et al., 2002] Brill, E., Dumais, S., and Banko, M. (2002). An analysis of the AskMSR question-answering system. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002), pages 257-264. Association for Computational Linguistics.

[Buscaldi and Rosso, 2006] Buscaldi, D. and Rosso, P. (2006). Mining knowledge from wikipedia for the question answering task. In In Proceedings of the 5th International Conference on Language Resources and Evaluation (LREC'06, page 727-730.

[Cabrio et al., 2012] Cabrio, E., Cojan, J., Aprosio, A. P., Magnini, B., Lavelli, A., and Gandon, F. (2012). Qakis : an open domain qa system based on relational patterns.

[Cao et al., 2011] Cao, Y., Liu, F., Simpson, P., Antieau, L., Bennett, A., Cimino, J. J., Ely, J., and Yu, H. (2011). AskHERMES : An online question answering system for complex clinical questions. Journal of Biomedical Informatics, 44(2) :277-288.

[Chen et al., 2016] Chen, D., Bolton, J., and Manning, C. D. (2016). A thorough examination of the cnn/daily mail reading comprehension task. CoRR, abs/1606.02858.

[Chen et al., 2017] Chen, D., Fisch, A., Weston, J., and Bordes, A. (2017). Reading wikipedia to answer open-domain questions. arXiv preprint arXiv :1704.00051.

[Chen et al., 2001] Chen, K., Kvasnicka, V., Kanen, P., and Haykin, S. (2001). Feedforward neural network methodology. IEEE Transactions on Neural Networks, 12(3) :647-648.

[Chollet et al., 2015] Chollet, F. et al. (2015). Keras.

[Craswell, 2009a] Craswell, N. (2009a). Mean Reciprocal Rank, pages 1703-1703. Springer US, Boston, MA. [Craswell, 2009b] Craswell, N. (2009b). R-Precision, pages 2453-2453. Springer US, Boston, MA. [Craswell, 2009c] Craswell, N. (2009c). R-Precision, pages 2453-2453. Springer US, Boston, MA.

[Dalip et al., 2013] Dalip, D. H., Gonçalves, M. A., Cristo, M., and Calado, P. (2013). Exploiting user feedback to learn to rank answers in q&a forums. In Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval - SIGIR '13. ACM Press.

[Devlin et al., 2018] Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). Bert : Pre-training of deep bidirectional transformers for language understanding.

[Diekema et al., 2004] Diekema, A. R., Yilmazel, O., and Liddy, E. D. (2004). Evaluation of restricted domain question-answering systems. In Proceedings of the Conference on Question Answering in Restricted Domains, pages 2-7, Barcelona, Spain. Association for Computational Linguistics.

48

Bibliographie

[ElKafrawy et al., 2018] ElKafrawy, P. M., Sauber, A. M., and Sabry, N. A. (2018). Semantic question answering system using dbpedia. In Lecture Notes in Computer Science, pages 821-832. Springer International Publishing.

[Fader et al., 2014] Fader, A., Zettlemoyer, L., and Etzioni, O. (2014). Open question answering over curated and extracted knowledge bases. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '14, page 1156-1165, New York, NY, USA. Association for Computing Machinery.

[Ferrucci et al., 2010] Ferrucci, D., Brown, E., Chu-Carroll, J., Fan, J., Gondek, D., Kalyanpur, A., Lally, A., Murdock, J. W., Nyberg, E., Prager, J., Schlaefer, N., and Welty, C. (2010). Building watson : An overview of the deepqa project. AI Magazine, 31 :59-79.

[Garg et al., 2019] Garg, S., Vu, T., and Moschitti, A. (2019). Tanda : Transfer and adapt pre-trained transformer models for answer sentence selection.

[Gazan, 2016] Gazan, R. (2016). Seven words you can't say on answerbag. In Proceedings of the 27th ACM Conference on Hypertext and Social Media - HT '16. ACM Press.

[Green et al., 1961] Green, B. F., Wolf, A. K., Chomsky, C., and Laughery, K. (1961). Baseball. In Papers presented at the May 9-11, 1961, western joint IRE-AIEE-ACM computer conference on - IRE-AIEE-ACM '61 (Western). ACM Press.

[Greff et al., 2016] Greff, K., Srivastava, R. K., Koutn'ýk, J., Steunebrink, B. R., and Schmidhuber, J. (2016). Lstm : A search space odyssey. IEEE transactions on neural networks and learning systems, 28(10) :2222- 2232.

[Harabagiu et al., 2001] Harabagiu, S., Moldovan, D., Pasca, M., Mihalcea, R., Surdeanu, M., Bunescu, R., Girju, R., Rus, V., and Morarescu, P. (2001). Falcon: Boosting knowledge for question answering.

[Hermann et al., 2015] Hermann, K. M., Kocisk'y, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., and Blunsom, P. (2015). Teaching machines to read and comprehend. CoRR, abs/1506.03340.

[Hewlett et al., 2016] Hewlett, D., Lacoste, A., Jones, L., Polosukhin, I., Fandrianto, A., Han, J., Kelcey, M., and Berthelot, D. (2016). WikiReading : A novel large-scale language understanding task over Wikipedia. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1 : Long Papers), pages 1535-1545, Berlin, Germany. Association for Computational Linguistics.

[Hill et al., 2015] Hill, F., Bordes, A., Chopra, S., and Weston, J. (2015). The goldilocks principle : Reading children's books with explicit memory representations. CoRR, abs/1511.02301.

[Hochreiter, 1998] Hochreiter, S. (1998). The vanishing gradient problem during learning recurrent neural nets and problem solutions. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 06(02) :107-116.

49

Bibliographie

[Iyyer et al., 2014] Iyyer, M., Boyd-Graber, J., Claudino, L., Socher, R., and DauméIII, H. (2014). A neural network for factoid question answering over paragraphs. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 633-644, Doha, Qatar. Association for Computational Linguistics.

[Jain et al., 2018] Jain, A., Kulkarni, G., and Shah, V. (2018). Natural language processing. International Journal of Computer Sciences and Engineering, 6 :161-167.

[Kaisser, 2005] Kaisser, M. (2005). Qualim at trec 2005 : Web-question answering with framenet. In TREC.

[Katz et al., 2002] Katz, B., Felshin, S., Yuret, D., Lin, J., Marton, G., McFarland, A., and Temelkuran, B. (2002). Omnibase : Uniform access to heterogeneous data for question answering. pages 230-234.

[Kolomiyets and Moens, 2011a] Kolomiyets, O. and Moens, M.-F. (2011a). A survey on question answering technology from an information retrieval perspective. Information Sciences, 181(24) :5412-5434.

[Kolomiyets and Moens, 2011b] Kolomiyets, O. and Moens, M.-F. (2011b). A survey on question answering technology from an information retrieval perspective. Information Sciences, 181(24) :5412-5434.

[Kowsari et al., 2019] Kowsari, K., Meimandi, K. J., Heidarysafa, M., Mendu, S., Barnes, L. E., and Brown, D. E. (2019). Text classification algorithms : A survey. CoRR, abs/1904.08067.

[Kumaran and Carvalho, 2009] Kumaran, G. and Carvalho, V. R. (2009). Reducing long queries using query quality predictors. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '09, page 564-571, New York, NY, USA. Association for Computing Machinery.

[Kwok et al., 2001] Kwok, C., Etzioni, O., and Weld, D. S. (2001). Scaling question answering to the web. ACM Trans. Inf. Syst., 19(3) :242-262.

[Lan et al., 2019] Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., and Soricut, R. (2019). Albert: A lite bert for self-supervised learning of language representations.

[Lee et al., 2006] Lee, M., Cimino, J., Zhu, H., Sable, C., Shanker, V., Ely, J., and Yu, H. (2006). Beyond information retrieval--medical question answering. AMIA Annu Symp Proc, pages 469-473.

[Leskovec et al., 2014] Leskovec, J., Rajaraman, A., and Ullman, J. D. (2014). Finding Similar Items, page 68-122. Cambridge University Press, 2 edition.

[Lopez et al., ] Lopez, V., Motta, E., Sabou, M., and Fernandez, M. Poweraqua : A multi-ontology based question answering system-v1.

[Loshchilov and Hutter, 2017] Loshchilov, I. and Hutter, F. (2017). Decoupled weight decay regularization. arXiv preprint arXiv :1711.05101.

[Mervin, 2013] Mervin, R. (2013). An overview of question answering system. 1.

50

Bibliographie

[Moldovan et al., 1999] Moldovan, D., Harabagiu, S., Pasca, M., Mihalcea, R., Goodrum, R., Girju, R., and Rus, V. (1999). Lasso : A tool for surfing the answer net.

[Moldovan et al., 2003] Moldovan, D., Pasca, M., Harabagiu, S., and Surdeanu, M. (2003). Performance issues and error analysis in an open-domain question answering system. ACM Transactions on Information Systems (TOIS), 21(2) :133-154.

[Molla Aliod and Gonz'alez, 2007] Molla Aliod, D. and Gonz'alez, J. (2007). Question answering in restricted domains : An overview. Computational Linguistics, 33 :41-61.

[Nam et al., 2009] Nam, K. K., Ackerman, M. S., and Adamic, L. A. (2009). Questions in, knowledge in? In Proceedings of the 27th international conference on Human factors in computing systems - CHI 09. ACM Press.

[Ojokoh and Adebisi, 2019] Ojokoh, B. and Adebisi, E. (2019). A review of question answering systems. Journal of Web Engineering, 17 :717-758.

[Olvera-Lobo and Gutiérrez-Artacho, 2011] Olvera-Lobo, M.-D. and Gutiérrez-Artacho, J. (2011). Multilingual question-answering system in biomedical domain on the web : An evaluation. In Multilingual and Multimodal Information Access Evaluation, pages 83-88. Springer Berlin Heidelberg.

[Olvera-Lobo and Gutiérrez-Artacho, 2011] Olvera-Lobo, M. D. and Gutiérrez-Artacho, J. (2011). Open- vs. restricted-domain qa systems in the biomedical field. Journal of Information Science, 37 :152-162.

[Pan and Yang, 2010] Pan, S. J. and Yang, Q. (2010). A survey on transfer learning. ieee transactions on knowledge and data engineering. 22 (10) : 1345, 1359.

[Paszke et al., 2019] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019). Pytorch : An imperative style, high-performance deep learning library. In Advances in neural information processing systems, pages 8026-8037.

[Qaiser and Ali, 2018] Qaiser, S. and Ali, R. (2018). Text mining: Use of tf-idf to examine the relevance of words to documents. International Journal of Computer Applications, 181.

[Radford et al., 2018] Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. (2018). Improving language understanding by generative pre-training.

[Rajpurkar et al., 2016] Rajpurkar, P., Zhang, J., Lopyrev, K., and Liang, P. (2016). Squad: 100, 000+ questions for machine comprehension of text. CoRR, abs/1606.05250.

[Reddy and Madhavi, 2017] Reddy, A. C. O. and Madhavi, K. (2017). A survey on types of question answering system. IOSR-JCE, 19(6) :19-23.

[Richardson et al., 2013] Richardson, M., Burges, C. J., and Renshaw, E. (2013). MCTest : A challenge dataset for the open-domain machine comprehension of text. In Proceedings of the 2013 Conference on Empiri-

51

Bibliographie

cal Methods in Natural Language Processing, pages 193-203, Seattle, Washington, USA. Association for Computational Linguistics.

[Robertson and Zaragoza, 2009] Robertson, S. and Zaragoza, H. (2009). The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends in Information Retrieval, 3 :333-389.

[Rodrigues and Milic-Frayling, 2009] Rodrigues, E. M. and Milic-Frayling, N. (2009). Socializing or knowledge sharing? In Proceeding of the 18th ACM conference on Information and knowledge management - CIKM '09. ACM Press.

[Ryu et al., 2014] Ryu, P.-M., Jang, M.-G., and Kim, H.-K. (2014). Open domain question answering using wikipedia-based knowledge model. Inf. Process. Manage., 50(5) :683-692.

[Sak et al., 2014] Sak, H., Senior, A. W., and Beaufays, F. (2014). Long short-term memory recurrent neural network architectures for large scale acoustic modeling.

[Seo et al., 2016] Seo, M., Kembhavi, A., Farhadi, A., and Hajishirzi, H. (2016). Bidirectional attention flow for machine comprehension.

[Shortliffe, 1977] Shortliffe, E. (1977). Mycin : A knowledge-based computer program applied to infectious diseases*. Proceedings / the ... Annual Symposium on Computer Application [sic] in Medical Care. Symposium on Computer Applications in Medical Care.

[Sun et al., 2015] Sun, H., Ma, H., Yih, W.-t., Tsai, C.-T., Liu, J., and Chang, M.-W. (2015). Open domain question answering via semantic enrichment. In Proceedings of the 24th International Conference on World Wide Web, WWW '15, page 1045-1055, Republic and Canton of Geneva, CHE. International World Wide Web Conferences Steering Committee.

[Tay et al., 2018] Tay, Y., Tuan, L. A., and Hui, S. C. (2018). Hyperbolic representation learning for fast and efficient neural question answering. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining - WSDM '18. ACM Press.

[Teufel, 2007] Teufel, S. (2007). An Overview of Evaluation Methods in TREC Ad Hoc Information Retrieval and TREC Question Answering, volume 37, pages 163-186.

[Ting, 2010] Ting, K. M. (2010). Precision and Recall, pages 781-781. Springer US, Boston, MA.

[Vargas-Vera and Motta, 2004] Vargas-Vera, M. and Motta, E. (2004). Aqua - ontology-based question answering system. In Monroy, R., Arroyo-Figueroa, G., Sucar, L. E., and Sossa, H., editors, MICAI 2004 : Advances in Artificial Intelligence, pages 468-477, Berlin, Heidelberg. Springer Berlin Heidelberg.

[Vaswani et al., 2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need.

52

Bibliographie

[Voita et al., 2019] Voita, E., Talbot, D., Moiseev, F., Sennrich, R., and Titov, I. (2019). Analyzing multi-head self-attention : Specialized heads do the heavy lifting, the rest can be pruned. arXiv preprint arXiv :1905.09418.

[Voorhees and Tice, 2000] Voorhees, E. and Tice, D. (2000). The trec-8 question answering track evaluation. Proceedings of the 8th Text Retrieval Conference.

[Wang et al., 2018] Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., and Bowman, S. (2018). GLUE : A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP : Analyzing and Interpreting Neural Networks for NLP, pages 353-355, Brussels, Belgium. Association for Computational Linguistics.

[Wong, 2007] Wong, W. (2007). Practical approach to knowledge-based question answering with natural language understanding and advanced reasoning.

[Woods, 1973] Woods, W. A. (1973). Progress in natural language understanding. In Proceedings of the June 4-8, 1973, national computer conference and exposition on - AFIPS '73. ACM Press.

[yacine, 2018] yacine, D. (2018). Recherche d'information: Prétraitement et indexation.

[Yang et al., 2015] Yang, Y., Yih, W.-t., and Meek, C. (2015). WikiQA : A challenge dataset for open-domain question answering. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2013-2018, Lisbon, Portugal. Association for Computational Linguistics.

[Yang et al., 2019] Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., and Le, Q. V. (2019). Xlnet : Generalized autoregressive pretraining for language understanding.

[Yoon et al., 2019a] Yoon, S., Dernoncourt, F., Kim, D., Bui, T., and Jung, K. (2019a). A compare-aggregate model with latent clustering for answer selection. pages 2093-2096.

[Yoon et al., 2019b] Yoon, S., Dernoncourt, F., Kim, D. S., Bui, T., and Jung, K. (2019b). A compare-aggregate model with latent clustering for answer selection. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM '19, page 2093-2096, New York, NY, USA. Association for Computing Machinery.

[Yu et al., 2018] Yu, A. W., Dohan, D., Luong, M.-T., Zhao, R., Chen, K., Norouzi, M., and Le, Q. V. (2018). Qanet : Combining local convolution with global self-attention for reading comprehension.

[Zhang et al., 2020] Zhang, Z., Yang, J., and Zhao, H. (2020). Retrospective reader for machine reading comprehension.

[Zheng, 2003] Zheng, Z. (2003). Question answering using web news as knowledge base. In Proceedings of the Tenth Conference on European Chapter of the Association for Computational Linguistics - Volume 2, EACL '03, page 251-254, USA. Association for Computational Linguistics.

53