Clustering text samples parameterised by the keywords of their elements

Abstract

This paper describes the solution to the problem of automated clustering of large text samples of a fixed length. Automatic grouping of texts of similar meaning is one of the most important tasks of data analysis since it has a wide scope of applications. The study focuses on the execution speed of the algorithm. That is why we consider a method to present a text sample by using its keywords as a set of document characteristics. Keywords are defined by the pre-computed values of the statistical TF-IDF measures. The next step involves text sample clustering. The study uses a modification of the Dbscan method, which is a density-based spatial clustering algorithm with the presence of noise. However, here it is interpreted as a form of breadth-first traversal with some restrictions of the document selection graph. Dbscan takes an index structure as an argument. This index structure is an object of links between documents in the corpus. Such an approach to the solution of the problem was chosen due to its presumed speed. To test this assumption, we measured the execution time of main operations, whose values are given to illustrate the test result of the proposed clustering method.