cURL is a CLI tool for issuing HTTP GET, POST, & PUT request.
1
vote
0answers
34 views
wget - How do I download webpages based on URL Pattern?
Consider a website www.music.com with the following directory structure:
/piano
/covers
/Chopin
apple.html
bannan.js
balloon.html
index.html
/Franz Liszt
...
1
vote
0answers
23 views
Rotate IPv6 for curl commands?
I have a bash script that needs to run curl several thousand times against different URLs.
Each time curl is run, I want to use a random IPv6 address from the /64 that's assigned to my VPS.
I read ...
0
votes
1answer
51 views
How to find out how is curl getting its proxy
I have a SUSE Linus and when I run curl it is using a proxy (without any proxy-related parameters passed).
Where can I find the proxy configuration?
I checked the http_proxy/https_proxy environment ...
0
votes
1answer
34 views
How to ignore the cookie path when using `cURL` request
I have two web-services
https://localhost/svc/auth which I call first with Authorization headers and I get a cookie
https://localhost/svc/catalog/1234 which I call by providing the cookie ...
-2
votes
0answers
39 views
yaourt - curl error
I have problem while trying to perform
yaourt -Syua
After Synchronizing package databases I've got this
curl error: Couldn't connect to server
I've tried disabling ipv6 in grub, adding aur....
1
vote
1answer
29 views
Getting empty data while reading text file and send read data to url by cURL in linux
I have text file called read_data.txt in /var/www/html/ directory. I want to read data by curl while file contents will change. So I have created a bash script. It is reading data while the content is ...
0
votes
1answer
43 views
Does the unix Curl command cache pages?
I was using TOR and used an API called ipify. (https://api.ipify.org). I tried running the command curl 'https://api.ipify.org?format=json' 2>&1 | grep -oP '\d+\.\d+\.\d+\.\d+' while connected ...
1
vote
1answer
29 views
Variable blank when used in if statement but not echo
I'm trying to save value from a curl command to a variable in a bash script.
Script looks like this
#!/bin/bash
curr=$(pwd)
IP_addr="192.168.0.102"
username="root"
password="pass"
HTTP_STATUS=$(...
0
votes
0answers
47 views
Run time for curl command
I have a file of domain names (~9000 urls). I am trying to determine the time taken to execute a successful curl request for all the urls.
#!/bin/bash
#read lines from file.
while read -r line || [...
1
vote
0answers
27 views
Extract contents of jar file to curl using scp
I have a jar file containing a javadoc tree on one box. I need to transfer it to another box and unpack the jar in a specific directory (which may not exist).
I have the username/password of the ...
0
votes
2answers
37 views
Evaluate command stored in Environment variable [duplicate]
A bash script I have limited or no ability to modify is using an environment variable like:
"${SERVER_ADDRESS}"
The actual SERVER_ADDRESS value can only be determined dynamically at runtime so I ...
0
votes
1answer
28 views
S3 head object but signature does not match
I want to use curl to head objects in s3.
code:
s3key=xxxxx... (s3 key)
s3skey=xxxxxx...(secret key)
bucket=xxx
file=/xx.txt
date=$(date +"%a, %d %b %Y %T %z")
string="HEAD\n\n\n$date\n/$bucket$file"...
0
votes
2answers
31 views
Convert cURL multi-line output to single, semi-colon-separated line
I'm using curl -s -I -L example.com | grep 'HTTP\|Location' to track re-directs of a given URL which gives multi-line output.
$ curl -s -I -L google.com | grep 'HTTP\|Location'
HTTP/1.1 301 Moved ...
-1
votes
2answers
87 views
using curl and shell script to restart apache
Iam want to restart the apache server, when its down.Hence i wrote like script below using curl and some if loop
curl example.com
si='curl example.com'
if test si !=0
then
service apache2 restart
fi
~...
0
votes
1answer
37 views
Can I make curl or wget restart if disconnected?
I'd like to download a file through curl - the problem is that my connection is very slow, and by unknown means, stops receiving data after ~180 kilobytes.
Unless there's a better method to fix it, I'...
-2
votes
1answer
53 views
CuRL command is not giving output as Url is not responding
when url is not responding curl command does not give any output/error because of this script does not move to next line.
command:
/usr/bin/curl --silent --ntlm --user 'username' --url http://server....
1
vote
0answers
39 views
Cron running tasks at second 26?
I've many vps with the same cron task sending data with curl every minute, e.g. date command output.
So I'm receiving datetime from every vps and I just realized that one of them is running the cron ...
0
votes
0answers
19 views
Curl website hidden behind loading screen
Some websites (example) have a "loading" screen display before they show you their actual content.
If I curl the site, I get this loading screen rather than the content which I want.
How can a bash ...
0
votes
1answer
16 views
upload x bytes using ftp/http
Is it possible to upload a specific number of bytes (binary data) to http/ftp server using linux command?
1
vote
1answer
63 views
How to POST 'shell output' as JSON data with Curl
I got the following command:
curl -H 'Content-Type: application/json' -X POST -d '{"host": "'$(hostname)'"}' http://sitename.com/update.php
Which works as expected, but if I try to send uptime ...
1
vote
1answer
55 views
php: compiling with openssl, ftp, ldap, curl support in debian gnu/linux
I'm trying to compile php 5.2.x in debian gnu/linux:
./configure --with-ldap --enable-ftp --with-apxs2 --with-mcrypt
--enable-bcmath --with-bz2 --enable-calendar --enable-dba=shared --enable-exif ...
1
vote
2answers
78 views
curl, wget do not return anything
I am trying this curl -I zomato.com | head -n 1
and I am not getting any response.
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed
0 0 0 ...
0
votes
1answer
46 views
curl download works fine but wget fails using ssl
When I try to download a file using curl command, the file is downloaded successfully:
curl -O https://domain/file --capath /etc/ssl/certs/
% Total % Received % Xferd Average Speed Time ...
1
vote
2answers
187 views
Curl - Argument list too long
My Question is similar to cUrl : Argument list too long
I am sending the following command to CURL from unix shell script
var=$(base64 sample.pdf | perl -pe 's/\n//g')
var1=$(curl -XPUT 'http://...
1
vote
1answer
46 views
Linux TCP stack not forwarding data to my application
I am using curl to send a HTTP request but curl app is not receiving response.
I tried to debug it using tcpdump and found that remote server is responding with data (HTTP 302 Found) but my curl ...
1
vote
0answers
47 views
cURL and wget cannot resolve hosts, but apt-get works
I've installed Ubuntu server 16.04 in a VirtualBox. If I try to install software with apt-get it works fine, but curl test.de or wget get.typo3.org/8.4.0 shows me that the host could not be resolved. ...
0
votes
1answer
23 views
curl -L not working on openSUSE Issue with Certificate
For some strange reason I can't execute curl -L https://github.com/Blosc/c-blosc/archive/v1.8.1.tar.gz. I need it for some package installation of Blosc that utilizes curl.
I get the error:
curl: (...
3
votes
2answers
374 views
Pipe the result of a cut command to curl
I have a csv file that contains 3 fields per line.
firstname,lastname,url
I'm trying to access the url via the following pipeline:
grep theName file.csv | cut -d, -f 3
then I want to add another ...
1
vote
1answer
134 views
Log Curl Output
I am trying to log my curl output which looks as following: (which are randomly produced every 10th sec)
<RTW>
<RESULT>{ "Result": { "something1": 707, "something2": { "array": ["95", "...
1
vote
0answers
16 views
some characters show after put objects on amazon s3
I use libcurl to put objects on amazon s3 service. Here is my original contents in a.txt:
aaaaaaaaaaaaaaaaaaaaaaaaaaa
bbbbbbbbbbbbbbbbbbbbbbbbbbb
...
17
votes
1answer
2k views
Why won't curl download this link when a browser will?
I am running Mac OS 10.11.6 El Capitan. There is a link I would like to download programmatically:
https://dev.mysql.com/get/Downloads/MySQL-5.7/mysql-5.7.16-osx10.11-x86_64.dmg
If I paste this URL ...
0
votes
2answers
85 views
How to grab the head section of a website using the curl and grep command?
I'm taking my Linux+ midterm and I'm stuck on a question where I'm supposed to copy the head section including the tags and output it to a file. My midterm is due on the 5th so it would really help if ...
-4
votes
1answer
45 views
CURL Operation failed
I want to make this upload to my server:
curl -u ****:**** -T - sftp://*****.com:22/apps/jboss-5.1.0/server/fab_server/deploy -v -k
The problem is that I'm getting this error:
% Total % ...
1
vote
1answer
206 views
What's the difference between using netcat (nc) and curl for HTTP requests?
I'm using curl to request a specific URL and getting 200 OK response:
curl -v www.youtypeitwepostit.com
* About to connect() to www.youtypeitwepostit.com port 80 (#0)
* Trying 54.197.246.21...
* ...
1
vote
1answer
33 views
Match a number with fixed digits while scraping web content
I'm trying to parse a source web pages, trying to find all href similar to this:
href='http://example.org/index.php?showtopic=509480
where the number after showtopic= is random (and with 6 fixed ...
2
votes
1answer
301 views
compiling curl with openssl giving wrong openssl version
I have found various answers on this, but I can't get anything that works for me. I am trying to compile curl with nghttp2 on my CentOS server.
I have compiled it all ok, but my problem is that curl -...
0
votes
2answers
41 views
Long Delay Before Curl Download [closed]
I have a bash script that checks to see if a file exists in a web directory. If the file doesn't exist, it simply quits. If the file does exist in the web directory, it downloads it.
The script works ...
0
votes
2answers
58 views
Upgrading curl but 'openSUSE-12.3-Update-Non-Oss' appears to be outdated
My webserver is running a Magento 1.8.1.0 that uses php-curl under the hood.
It runs openSUSE 12.3 and:
PHP 5.3.17
Apache 2.2.29
mysql Ver 14.14 Distrib 5.5.33
OpenSSL 1.0.1j
The curl version ...
1
vote
2answers
138 views
“unable to get local issuer certificate” and “tlsv1 unrecognized name”
I am having a problem with the Paypal IPN postback verification. The problem came up a month ago, I believe when Paypal updated their security standards and their certificates.
My website is running ...
1
vote
1answer
63 views
Piping multiple inputs into Ghostscript
I am trying to create a one-line Linux command to combine two PDF files - which are downloaded from a URL - using Ghostscript. However, I do not want to create any temporary files (everything should ...
0
votes
0answers
115 views
gnome software not working on fedora 24
Gnome software on fedora 24 is not working. It shows
Detailed errors from the package manager follow:
Curl error (28): Timeout was reached for https://mirrors.fedoraproject.org/metalink?repo=fedora-...
3
votes
1answer
1k views
Uploading multiple files via ftp using curl
I'm trying to upload all the text files within the current folder via FTP to a server location using CURL. I tried the following line:
curl -T "{file1.txt, file2.txt}" ftp://XXX --user YYY
where ...
2
votes
1answer
65 views
Parsing HTML with cURL POST
I'm sending POST request to some link with valid username and password.
I use this command: curl -X POST -F '[email protected]' -F 'password=123' http://my.site.com/login
but it got Moved ...
0
votes
1answer
79 views
How to download an ISO and directly create a bootable USB
How can I transfer an .iso file to an USB in parallel to the download of this file so that downloaded data gets directly on my USB without passing through my hard drive.
0
votes
0answers
19 views
Storing the html page of the redirected page using curl after submitting the form details
I am using curl for sending the form details and I want to store the redirected page in the file. But the html code of the home page gets stored rather than the redirected page.
curl -X POST -d "...
2
votes
1answer
56 views
Docker file fail to cd into a directory
Using this this Dockerfile , line 21 gives the error:
/bin/sh: 1: cd: can't cd to /home/meteorapp/meteorapp/app
Any idea how to go about fixing this?
2
votes
0answers
260 views
Fedora 24 : Gnome Software Center and Curl not working behind proxy servers
I have Fedora 24 Workstation on my laptop. My internet access is through proxy server only.
I have successfully configured proxy settings for dnf and wget. I can install and updates packages using ...
1
vote
1answer
33 views
Help me understand variable place holder for curl command in Dockerfile
I am learning about Dockerfile and had not worked with curl before.
I am trying to understand line 26 in this Dockerfile .
&& bash -c 'curl "https://nodejs.org/dist/$(<.node_version....
3
votes
2answers
61 views
Extract a value from command output and use the value as a parameter to a subsequent command
Hi there I am trying to have
curl -v --request POST https://example.com --data-urlencode "username=usr]” --data-urlencode "password=[pass]”
pass the output to the [TOKEN] field in here:
curl -D - --...
2
votes
3answers
147 views
Merge jq output into a comma seperated string [duplicate]
I am trying to curl some URL which return json file, then I want to parse hosts from it and create a comma separated string.
I have the first part working
curl -s -u "admin:admin" -H "X-Requested-...
