If, for example, to take a relational base, then in the table it is necessary to find records that depend on each other in a certain way. Combinations of multiple entries may depend on other combinations of multiple entries. Records can be millions.
In a relational DBMS, this can be accomplished using a certain number of join. But since there can be millions of records, after the first join, the DBMS will have to deal with a trillion records, etc. And the request in this case can be processed indefinitely (already checked).
Probably there is, intended just for such tasks?
While looking at graph DBMS (neo4j), but have not yet figured out, it will fit or not. Advise in which direction at least to move.

  • 3
    after the first join, the DBMS will have to deal with a trillion records, etc. No need to talk about what the DBMS will do if you do not know what and how it does. - Akina
  • Checked by whom, by what exactly query and which DBMS. Why do you think that you were able to make a perfect query for your task and actually build a database suitable for its solution? And why do you think that you need to combine all the data using join, there are different approaches to solving the same problems. And where do trillions of records come from, if the selection goes according to a well-constructed index, then join it right away it takes the records that match exactly your condition, without revising the entire set. - Mike
  • And by the way, I told you about a trillion, but this was in response to a single comment that related to one particular join. And it meant only that it was necessary to look for another solution, not fruiting numerous join in the query. Perhaps it was necessary to apply some sort of window functions or a recursive query, but so that it would work by index. - Mike
  • PS I personally would not consider DBMS written in java at all. because java and fast processing of millions of records are not compatible - Mike
  • 2
    To the question " Advise in which direction at least to move, " I can answer that it is necessary to move towards accurate formulation and formalization of the task. You already ask 3 consecutive questions, but have not clearly described the task. Unable to find a means of solving the problem without understanding it to the end - Mike

0