Models of mixed integer linear programming optimization of solutions for distributed data storage and processing

Abstract

Modern methods for identifying efficient solutions for distributed storage and distributed data processing do not provide for optimization of the performance of computing devices allocated to each process implementing data operations, depending on the deadlines and budgets for their execution specified by users. In this regard, two mathematical models of mixed integer linear programming have been developed that allow for optimization of distributed storage and data processing (including optimization of the performance of computing devices allocated to each processing process), as well as data transfer between devices. These models implement the identification of optimal solutions under the condition of minimizing the total cost of performing operations (taking into account the restrictions on the deadlines for obtaining results and the total performance of devices), as well as minimizing the total duration of implementing these operations (taking into account the restrictions on the budgets for data processing and the performance of devices). The initially obtained nonlinear models are linearized by introducing additional variables. The developed models are implemented programmatically using the OR-Tools package for solving discrete optimization problems, which allows interpreting the target functions and constraints of the specified models using its library and the Phython language. The developed application allowed us to study the process of distributed storage, transmission and processing of data with different values ​​of the parameters that characterize it. Based on the analysis of the obtained research results, conclusions were formulated regarding the dependence of the total duration of operations on the values ​​of the directive deadlines with different values ​​of the number of operations performed with data, as well as the dependence of the total duration of data processing on the specified budgets and the cost of computing operations and data storage.