Those. we are talking about URLs that are technical in nature, and that the user does not need to show

    1 answer 1

    from robots to robots.txt:

    Disalow:/your/ajax/url

    From the user, the meaning of this action does not justify its implementation. So how do you request and the browser keeps track of it. Conclusion:

    1. From robots worth closing.
    2. From users do not even bother
    • I add that robots.txt does not close the page from indexing. Just search robots do not go through these pages. In general, this is done to reduce server load. And to close the indexing you need to send a header or a meta tag to insert. Just reminded about it! - user208916