Hello!

I need to download from Vkontakte 100 posts from 30 thousand communities. Then I have to process this information. Advise what to use to save this data, then to be able to load it again most quickly?

Closed due to the fact that it is necessary to reformulate the question so that it was possible to give an objectively correct answer by the participants αλεχολυτ , aleksandr barakin , pavel , Streletz , Dmitriy Simushev 25 Sep '16 at 20:23 .

The question gives rise to endless debates and discussions based not on knowledge, but on opinions. To get an answer, rephrase your question so that it can be given an unambiguously correct answer, or delete the question altogether. If the question can be reformulated according to the rules set out in the certificate , edit it .

  • Regular files are not an option? - Vladimir Martyanov
  • 2
    The word big-data should not be used for data that can fit in the RAM on the phone. - jfs

2 answers 2

Start with the simplest: use SQLite for saving and subsequent queries, or even easier as throwing a json file onto a disk to drop (as firefox for session recovery does — minus compression).

Pickle and derived formats implicitly encourage more complex data structures and are less portable between program versions or other languages.

    Look towards the pickle module.

    A very simple and convenient way to save arbitrary data structures to a file, then read them and get the same structures.