
```shell
wget [options] <URL> # Basic usage
wget https://example.org # Download a web page to local disk
wget -O main.html https://example.org # Save the file as `main.html`
wget -P ~/src/docs/ <URL> # Save the file to `~/src/docs/` directory
wget -o <logfile> <URL> # Log all messages to logfile

wget --limit-rate=500k <URL> # Limit download speed to 500 KBps
wget -tries=100 <URL> # Specify retry attempts

wget -b <URL> # Download in background.
# If no output file is specified via the -o, output is redirected to `wget-log`

wget -i <file> # Read URLs to download from file
wget -w 20 -i <file> # Wait 20 seconds between retrievals

wget --ftp-user=<username> --ftp-password=<password> ftp://example.org/spam.tar # Download via FTP
wget -c https://example.org/archive.tar.gz # Continue interrupted downloads

wget -m -k -p -np -P ~/docs/web/ https://example.org # Download entire website
# -m, --mirror: Turn on options (recursion, time-stamping, etc) suitable for mirroring
# -k, --convert-links: Convert the links to make them suitable for local browsing
# -p, --page-requisites: Download all the files necessary for local browsing (stylesheets, images, sounds, etc)
# -np, --no-parent: Ensure that directories above the hierarchy are not retrieved
```

- Find broken links on your website [[\*](https://www.digitalocean.com/community/tutorials/how-to-find-broken-links-on-your-website-using-wget-on-debian-7)]

  ```shell
  wget --spider -r -nd -nv -H -l 1 -w 2 -o run1.log http://your_server_ip/index.html
  ```


Wget

[Top]


## GET

```shell
curl [options] <URL> # Basic usage
curl https://github.com/robots.txt # Display the contents of remote file
curl -O https://github.com/robots.txt # Save the file with the original filename
curl -o main.txt https://github.com/robots.txt # Save the file as `main.txt`
curl -L https://github.com/robots.txt # Follow redirects

curl https://api.github.com/repos/3b1b/manim # Basic GET request to the GitHub API
curl -i https://github.com/robots.txt # Include HTTP-header in the output
curl -I https://github.com/robots.txt # Fetch HTTP-header only
curl -C - -O http://example.org/archive.tar.gz # Resume interrrupted download

# -u, --user: Specify the user name and password for server authentication
curl -u <username:password> https://api.example.com/endpoint # Basic authentication
curl -u <username> https://api.example.com/endpoint # Prompt for password; Avoid having password in history

# -m, --max-time: Set maximum time in seconds that you allow the whole operation to take
curl -m 1800 <URL> # Set a 30 minute timeout
```

## POST

```shell
# -d, --data: Send data in a POST request
curl -d '{"name": "Professor Fansworth", "phone": "1234567890"}' <URL>
curl -d "name=Professor%20Fansworth&phone=1234567890" <URL> # Equivalent to above
curl -d @data.json <URL> # Read data to POST from file

# -H, --header: Specify headers
curl -H "Content-Type: application/json" -H "Authorization: <token>" -d @data.json <URL>
# Can be used multiple times to add/replace/remove multiple headers

# -b, --cookie: Pass the data to the HTTP server as a cookie
curl -b "name=Fry" <URL>
curl --dump-header <file> <URL> # Save cookies received from the server to file
curl -b <file> <URL> # Use cookies from file
curl -c <file> <URL> # Save the incoming cookies using Netscape cookie format

# -x, --proxy: Use the specified HTTP proxy and port number (default: 1080)
curl -u <username:password> -x <proxyhost[:port]> <URL>

# -U, --proxy-user: Specify the user name and password for proxy authentication
curl -U <username:password> -x <proxyhost[:port]> <URL>

# -v, --verbose: Make the fetching more verbose/talkative
# -s, --silent: Make curl silent
```


curl


## Check for existing keys

```shell
ls -al ~/.ssh
```

## Generate a new key

```shell
ssh-keygen -t ed25519 -C "your_email@example.org" # Recommended
ssh-keygen -t rsa -b 4096 -C "your_email@example.org" # For legacy systems
```

## Add your key to the ssh-agent

```shell
eval "$(ssh-agent -s)"
ssh-add ~/.ssh/<Private key file>
```
