How to Use Wget to Download Only Newer Files on Ubuntu

This guide explains how to use the wget command in Ubuntu to download files only when they are newer than your local copy. We will cover the specific timestamping flag required for this task, provide practical examples, and discuss how this feature saves bandwidth and time during repetitive download operations.

The Timestamping Flag

To download files only if they are newer than the local version, you must use the -N flag. This stands for timestamping. When this option is enabled, wget checks the timestamp of the remote file against the local file before downloading.

Command Syntax

The basic syntax for this operation is straightforward. Open your terminal and enter the following command:

wget -N http://example.com/path/to/file

You can also use the long form of the command if you prefer readability:

wget --timestamping http://example.com/path/to/file

How It Works

When you run this command, wget sends a request to the server to check the last modified date of the file. If the remote file is newer than the local copy, wget downloads the new version and overwrites the old one. If the local file is already up to date, wget skips the download entirely.

Important Considerations

For this feature to work correctly, the web server must send a valid “Last-Modified” header with the file. If the server does not provide this timestamp information, wget cannot compare the files and will default to downloading the file again. Additionally, this method works best with HTTP and FTP protocols where timestamp metadata is reliably available.

Saving Bandwidth

Using the timestamping flag is essential for automation scripts or cron jobs that need to keep files updated without redundant data usage. By ensuring only changed files are transferred, you reduce network load and speed up update processes on your Ubuntu system.