I am working on an Apache log parser. Parsing is performed by a Python 3 script hosted by Django. The log file is downloaded and parsed. Now the program can save parsed log data in SQLite models. I also need to save in RAM, and then make statistics on the parameters of records. I know Redis is a key: value repository. How to correctly write data in Redis, so that later it would be convenient to make queries for analysis on any field or on the aggregate of several fields?
1 answer
- For parsing logs jango is not needed, you can simply run the python script from the console and parse the logs.
- It is not clear why storing data in RAM, the logs can be large and there is not enough RAM
- In the database it is just convenient to analyze the data and connect the tables in different fields, as you will do it in radishes, I do not know
inzem77 inzem77
429 2 6
- Django, because it is a task; I assumed that it was necessary to do using caching, because it is a pity for SSD, besides, the processing speed does not hurt. I do not know yet which stack is better to use for this task. - Elefanobi
- RAM is more expensive than ssd. Take the usual disk space and save everything in the database - inzem77
- that's exactly what I did. - Elefanobi
|