How to remove duplicates without sorting preserving the order of the original list in a text document? Using linux or python. Or as an option: there is a list1 (not sorted), a list2 (not sorted). How to add a list2 to the end of a list1, but at the same time exclude duplicates from the list2 that are already in the list1 and the word order should remain the same. thank

  • It is impossible to do this using the “means” of the linux program: it is designed for completely different tasks (to serve as the kernel in a number of operating systems). but with the help of some other programs - easily - aleksandr barakin
  • Thanks, for some reason I did not find this question. More like $ awk '!a[$0]++' исходный-файл - TWOfish
  • Yes, the solution is interesting, but very costly in terms of the use of RAM when increasing the input file (in comparison with the solution I proposed, where the file size, as I understand it, does not affect the size of the memory used at all). - aleksandr barakin
  • @alexanderbarakin generally speaking, it should be faster (approaches O (N)) and less expensive in memory (O (n) versus O (N), where N is the number of lines in the file, and n is the number of unique strings) than a solution with sort'om ... - Fat-Zer
  • @ Fat-Zer, maybe. would test. yes laziness - aleksandr barakin

0