Questions tagged [wget]
GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive command line tool, so it may easily be called from scripts, Cron jobs, terminals without X Window System (X11) support, etc.
907
questions
0
votes
0
answers
22
views
What exactly is the point of WARC files with wget?
I am trying to archive my small advertisements from a small advertisement portal immediately after I have sold the respective article and before I delete the advertisement.
I know that I can't do that ...
2
votes
0
answers
40
views
Why is GNU wget only returning an index.html.tmp file, instead of the actual files I want?
I want to download a lot of satellite data (organized by each day of each year) in .hdf format from NASA's LAADS DAAC archive. They have a helpful guide that provides a code sample to use with GNU ...
1
vote
0
answers
23
views
How to skip a directory when downloading with wget?
I want to download a website with wget in Linux that have the following 'structure':
sssssssssss.com/things/summer/tables/books/
There is a directory in the directory 'summer', named 'sport'.
I want ...
0
votes
0
answers
43
views
Wget show error certificate of <domain> name was signed using an insecure algorithm, while curl works
We are trying to check domain
wget domain <like httpx://abc.com>
we are getting error "The certificate of domain(abc.com) was signed using an insecure algorithm. But SSL certificate is ...
0
votes
0
answers
45
views
Retrieve all ZIPs from specific site subdirectory
I'm trying to retrieve the *.ZIP files from this Zophar.net Music section, specifically the NES console. The files are freely downloadable per each game separately, which will be a huge time sink to ...
0
votes
0
answers
21
views
How to list a tree of webpage files given url
I can use wget --mirror url to download a webpage, but how can I only list (see) a tree directory of that page in linux terminal?
For example I have a personal website where there is a .htaccess file ...
0
votes
0
answers
50
views
How skip downloading 'robot.txt.tmp' files?
I sometimes want to only download media files from a single web page, such as gif files, which I figured out with:
wget -P c:\temp -A .gif -r -l 1 -H -nd 'https://marketplace.visualstudio.com/items?...
0
votes
1
answer
87
views
retrieve response body from wget --method=PUT
I knwo that with GNU wget (https://www.gnu.org/software/wget/manual/wget.html) I can send PUT HTTP requests with --method=PUT, but I cannot figure out how to read the response body I get back.
What I'...
0
votes
0
answers
110
views
How to find download url from a given website to be used in wget?
I try to obtain a publicly available historical document from an archive in Germany, however, the download never starts as I believe the PDF file is too big to be downloaded within a certain time, so ...
0
votes
0
answers
231
views
Inconsistent SSL connection error when using wget
I'm trying to download some .nc files from a secure online database.
It seems like for roughly every other attempted file download, I get an 'Unable to establish SSL connection' message, and then wget ...
0
votes
0
answers
45
views
YARN resource manager "HTTP request sent, awaiting response..."
we have Hadoop cluster with active/stand by resource manager services the active resource manager is on master1 machine and the stand by resource manager is on master2 machine
in our cluster YARN ...
-1
votes
1
answer
77
views
how to get back the file that has been downloaded using Invoke-Webrequest without providing the output switch?
I downloaded a large file, using wget which is just alias for Invoke-WebRequest commandlet in powershell, and it asked me url i provided, it completed, and gave me the Headers of HTTP request instead ...
2
votes
1
answer
648
views
Stop wget from making index.html for Options +Indexes directories
I am using this command to mirror a directory tree on my server:
wget -m -e robots=off --no-parent --reject-regex index.html --reject-regex aaa --no-verbose --timestamp https://example.com/folders/
&...
1
vote
1
answer
152
views
Getting "Cannot write to" message after successful mirroring of remote directory with wget
I have been trying to mirror a directory on my server using wget, then afterward update the downloaded tree with only newer files. The local machine is running MacOS 12.6.3; the server is CloudLinux, ...
0
votes
0
answers
130
views
WSL wget command returns HTML rather than downloadable csv file
I am running WSL (Windows Subsystem for Linux) and using Ubuntu 22.04 to test my homegrown bash scripts as a part of my CS course.
The task is to download a CSV (comma separated values) file from the ...