How to Check if Remote File Exists Before Wget Download on Ubuntu

This article explains how to verify the existence of a remote file before initiating a download using wget on Ubuntu. It covers command-line methods to prevent errors and save bandwidth by ensuring the target URL is valid prior to execution.

Use the Spider Command

The most effective way to check if a remote file exists without downloading it is by using the --spider flag with wget. This option makes wget act like a web spider; it checks if the file exists but does not save any data to your disk.

Run the following command in your terminal:

wget --spider https://example.com/file.zip

If the file exists, wget will display a message indicating that the remote file exists and return an exit code of 0. If the file does not exist or the URL is broken, it will report an error and return a non-zero exit code.

Using Exit Codes in Scripts

When automating tasks in bash scripts, you can rely on the exit status of the wget command to determine the next steps. A successful check returns 0, while a failure returns 8 or another non-zero value.

Here is an example of how to implement this check in a script:

if wget --spider -q https://example.com/file.zip; then
    echo "File exists. Starting download."
    wget https://example.com/file.zip
else
    echo "File does not exist. Skipping download."
fi

The -q flag stands for quiet mode, which reduces output clutter during the check. This method ensures your Ubuntu system only attempts downloads when the remote resource is confirmed to be available.