О ПОДХОДАХ ДЛЯ ОПРЕДЕЛЕНИЯ МЕРЫ НЕСХОДСТВА В ТЕКСТОВЫХ ДАННЫХ

Из-за стремительного роста текстовых данных исключительно важно обрабатывать ее. В статье рассмотрены различные способы получения меры сходства/несходства для текстовых данных. Представлены различные методы учитывающие лексическое сходство строк, а также семантическое расхождение текста.

1. Yunianta A. Semantic data mapping technology to solve semantic data problem on heterogeneity aspect / A. Yunianta, O. M. Barukab, N. Yusof, N. Dengen, H. Haviluddin, M. S. Othman // International Journal of Advances in Intelligent Informatics. – 2017. – vol. 3, no. 3. – pp. 161–172.

2. Hidayat E. Y. Automatic Text Summarization Using Latent Drichlet Allocation (LDA) for Document Clustering / E. Y. Hidayat , F. Firdausillah, K. Hastuti, I. N. Dewi, A. Azhari // International Journal of Advances in Intelligent Informatics. – 2015. – vol. 1, no. 3. – p. 132.

3. Hall P. A. V. Approximate string matching / Patrick A. V. Hall, Geoff R. Dowling // Computing Surveys. – 1980. – vol. 12 no. 4. – pp. 381–402.

4. Jaro, M. A. Advances in record linkage methodology as applied to the 1985 census of Tampa Florida / M. A. Jaro // Journal of the American Statistical Society. – 1989. – vol. 84, no. 406. – pp. 414-420.

5. Jaro, M. A. Probabilistic linkage of large public health data file / M. A. Jaro // Statistics in Medicine. – 1995. – vol. 14. – pp. 491–498.

6. Kondrak G. N-gram similarity and distance / G. Kondrak // International symposium on string processing and information retrieval. – 2005. – pp. 115–126.

7. Yu M. String similarity search and join: a survey / M. Yu, G. Li, D. Deng, J. Feng // Frontiers of Computer Science. – 2016. – vol.10, no. 3. – pp. 399–417.

8. Eugene F. K. Taxicab Geometry / F. K. Eugene. – Dover Publications, 1987. – p. 96

9. Dice L. R. Measures of the Amount of Ecologic Association Between Species / L. R. Dice // Ecology. – 1945. – vol. 26, no. 3. – pp. 297–302.

10. Lund K. Semantic and associative priming in high-dimensional semantic space / K. Lund // Proc. of the 17th Annual conferences of the Cognitive Science Society. – 1995, pp. 660–665.

11. Landauer T. K. A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge / T. K. Landauer, S. T. Dumais // Psychological Review. – 1997. – vol. 104, no. 2. – pp. 211–240

12. Gabrilovich E. Computing semantic relatedness using wikipedia-based explicit semantic analysis / E. Gabrilovich, S. Markovitch // IJcAI. – 2007. – vol. 7. – pp. 1606–1611

13. Mihalcea R. Corpus based and knowledge-based measures of text semantic similarity / R. Mihalcea, C. Corley, C. Strapparava // American Association for Artificial Intelligence. – 2006. – vol. 6. – pp. 775–780,

14. T. Slimani Description and Evaluation of Semantic Similarity Measures Approaches / T. Slimani // International Journal of Computer Applications. – 2013. – vol. 80, no. 10. – pp. 25–33

15. Tversky A. Features of similarity / A. Tversky // Psychological Review. – 1977. – vol. 84, no. 4. – pp. 327–352, 1977