How to Handle Unstable Connections and Resume Downloads with Wget

Wget is a powerful command-line utility used by server administrators to download files over HTTP, HTTPS, and FTP. It’s especially useful when working on servers without a graphical interface or when automating tasks via scripts.

In this guide, we’ll explore three common use cases that every server admin should know:

1. Handling unstable server connections.
2. Restarting a download without extra parameters.
3. Running wget in the background for long downloads.

Each section includes practical examples and tips to help you make the most of wget in real-world scenarios. 

1. Handling Unstable Server Connections

Sometimes, you need to download large files from a server with an unstable or unreliable connection. Without the right options, the download may fail repeatedly, wasting time and bandwidth.

Here’s a powerful command to deal with this situation:

wget -c --tries=0 --timeout=60 --waitretry=5 https://domain.com/bigfile.zip

Explanation of the flags

• -c (continue)

  • Allows wget to resume partial downloads.
  • If the connection drops, wget picks up where it left off instead of starting again from scratch.

• --tries=0

  • Sets the number of retry attempts to unlimited.
  • This is useful for very unstable servers where you need wget to keep trying until it succeeds.

• --timeout=60

  • Sets the network timeout to 60 seconds.
  • If no data is received for 60 seconds, wget will treat it as a failed attempt and retry.

• --waitretry=5

  • Adds a 5-second delay between retry attempts to avoid overloading the server or hitting rate limits.

Tip: You can adjust --waitretry depending on how aggressive you want the retries to be. A higher value is better for avoiding server strain.

Practical use case

Imagine you’re downloading a large backup file from a remote location where the connection frequently drops. This command will automatically retry indefinitely, pausing five seconds between attempts, and always resume where it left off.

2. Restarting a Download Without the Extra Flags

If you’ve already started downloading a file with wget but didn’t use the -c flag, you might worry about having to start over. Luckily, wget makes it simple to resume a download without typing the full command again.

Using the same command

When you first start a download, wget saves the file under its original name. To resume the same file, simply rerun wget with the same URL and add the -c flag:

wget -c https://domain.com/bigfile.zip

• If the partially downloaded file is present in the same directory, wget will detect it automatically and resume from where it stopped.
• If the file is complete, wget will verify the file size and skip re-downloading.

Note: If the server doesn’t support HTTP range requests, wget will start from the beginning even with the -c option.

Why this matters

This is especially helpful if:

• The original download was interrupted due to network issues.
• The server temporarily went offline.
• You closed your terminal session by mistake.

Tip: Always check the existing file before resuming by using:

ls -lh

This helps confirm the current file size and whether resuming makes sense.

3. Running Wget in the Background

Large downloads can take hours, and you don’t always want to keep a terminal session open while waiting. Wget offers a simple way to run as a background task, letting you disconnect safely without interrupting the download.

Using the -b flag

Here’s the command:

wget -b -c https://domain.com/bigfile.zip

• -b (background mode)

  • Starts the download in the background and immediately returns control to the terminal.
  • Perfect for situations where you need to log out of an SSH session but want the download to continue.

• -c (continue)

  • Ensures that if the download stops for any reason, it can be resumed later.

When you use -b, wget automatically creates a log file named wget-log in the current directory. You can monitor progress by checking this file:

tail -f wget-log

Tip: Use Ctrl + C to exit the log viewer without stopping the download.

Alternative: Running Wget with nohup

If you want more control or need to run wget inside a script, consider using nohup. This prevents the process from being terminated when you close the terminal.

nohup wget -c https://domain.com/bigfile.zip &

• nohup keeps the process running even if you log out.
• The & at the end places the task in the background.
• Output is saved to a file named nohup.out unless specified otherwise.

Practical Workflow Example

Here’s how a typical workflow might look when downloading a large file from an unreliable server:

1. Start the download with retry options:

   wget -c --tries=0 --timeout=60 --waitretry=5 https://domain.com/bigfile.zip
   

2. If the connection drops, resume later:

   wget -c https://domain.com/bigfile.zip
   

3. If you need to disconnect, move it to the background:

   wget -b -c https://domain.com/bigfile.zip
   

4. Monitor progress:

   tail -f wget-log
   

5. Verify the file once complete:

   ls -lh bigfile.zip
   

Additional Tips for Wget

Here are a few more useful wget options that can improve your workflow:

• Limit download speed to prevent server overload:

  wget --limit-rate=500k https://domain.com/bigfile.zip
  

  This limits the speed to 500 KB/s.

• Set a custom output filename:

  wget -O myfile.zip https://domain.com/bigfile.zip
  

  Helpful when the server’s default filename isn’t descriptive.

• Mirror a full website:

  wget --mirror --convert-links --adjust-extension --page-requisites https://example.com
  

  Ideal for backups or testing offline.

Warning: Always make sure you have permission before mirroring a site.

Further notes...

Wget is an essential tool for server administrators, especially when working with large files or unstable network conditions. By understanding a few key commands, you can:

• Handle unreliable servers with automatic retries and resume support.
• Restart interrupted downloads without starting over.
• Run downloads in the background, keeping your session free for other tasks.

These techniques save time, reduce frustration, and make file management far more efficient.

With the right options, wget becomes a reliable ally for any server admin looking to streamline downloads and keep workflows running smoothly.