There are 3 tables:

parsers

parsers -id -name -created_at

parsers_settings

 parsers_settings -id -parser_id -some_config_field

and for example amazon_parser_products

 amazon_parser_products -id -amazon_product_id -name

I need to somehow save information about the execution of each parser and link the products from the table "amazon_parser_products" to the execution in order to know when and at what execution the products were added from the parser. There is an option to add a table

 parsers_executions -id -status -finished_at

And then amazon_parser_products add another field

 amazon_parser_products -id -execution_id -amazon_product_id -name

Norm so do? or are there better options?

    1 answer 1

    In order to get the most correct answer, you would have to describe everything in the processes and the essence - this would give a clearer picture for the preparation of suitable tables.

    Based on what you describe, I would recommend that you store data in the following form:

     parsers - id - name - type - created

    Parsers, with an indication of the types of parsers

     parser_settings - id - parser_id - some_config_field

    Parser Settings

     products - id - product_type - external_id - name

    Products, indicating the type of product and external identifier. All imported products should be reduced to the same type (+ - several fields can be left empty for types where such data is missing, but you should not pursue all types of support for dubs). Otherwise, you can drown in many different types, which leads to the impossibility of using isomorphism and the need to write all the code again for each type.

     executions - id - status - finished_at

    Import tasks

     products_to_executions - product_id - execution_id

    Ratios of imported goods and tasks within which they were imported. It should be stored in a separate table, as the relationship between the entities Product and Task - many to many

    • thanks for the answer . Look, in the "products" table the goods from all parsers are stored. wouldn't it be better to store goods from each parser in a separate table? those. separate site = separate table - Kolya Vantukh
    • @KolyaVantukh And then what will you do with the goods? If they are in separate tables, for example, you will not be able to receive a complete list of products in the database in one request. And to store non-standard attributes, you can create a JSON type field or use the EAV structure (depending on your needs for working with them) - Mike
    • @KolyaVantukh as I described above, so you lose the possibility of polymorphism. For example, if you take out the list of all imported goods, then 1) you cannot take them from the database with one query, you will have to do it by a separate query in each table. 2) By requesting different products from different tables, you will have to store them in memory in different arrays and process them separately, since they will have different types (that is, a different set of fields) 3) You cannot easily sort them, only by separateness Therefore, it is better to bring all types to one type. - Dmitry Shilin