I download files with the wget crontab command from ftp and I need to copy the last 10 modified files to another directory, since I'm only interested in them, and leaving the entire set of xml files of ~ 1GB in size will be time consuming to process.

I also take into account that wget should complete, and only then copy it, i.e. sequential execution of console script commands.

 #!/bin/sh wget -конфигурация ; cp ...

Please help configure the copy command in my case.

  • I did not understand the problem. You need to download the files yourself, check them (ways to compare a lot) with the target ones and copy where necessary, right? And what exactly does not work? - approximatenumber
  • In fact, now I'm downloading files from ftp, because catalog is updated. Therefore, the downloaded files need to be copied to another directory and then somehow processed. - white-imp

1 answer 1

if the http server gives the correct last-modified header, then the wget program sets the file modification time specified in such a header.

You can get a list of files in the current directory sorted by modification time, for example, using the ls program with the -t option:

 $ ls -t *

newer files will be present higher older ones.

You can select the required number of lines from the beginning of the list, for example, using the head program, specifying the number using the -n количСство option:

 $ ls -t * | head -n количСство

the result can be substituted as a set of first arguments to the cp program (the last argument should be the directory where these files will be copied):

 $ cp $(ls -t * | head -n количСство) /ΠΊΡƒΠ΄Π°/ΠΊΠΎΠΏΠΈΡ€ΠΎΠ²Π°Ρ‚ΡŒ/

but if special characters are supposed to exist in the file names (for example, a space), then this command will have to be considerably more complicated, using, for example, the capabilities of the xargs program:

 $ ls -t * | head -n количСство | xargs -I'{}' -d '\n' cp '{}' /ΠΊΡƒΠ΄Π°/ΠΊΠΎΠΏΠΈΡ€ΠΎΠ²Π°Ρ‚ΡŒ

details:

  • $ man ls
  • $ man head
  • $ man cp
  • $ man xargs