curl

This function use library libcurl to retreive data from a URL.

CURL

NAME

curl - This function use library libcurl to retreive data from a URL.

SYNOPSIS

curl (url:"string" [,httppost:array,verifypeer:bool,error:variable, data:string, cookiefile:string, writefile:string, file:string, textfile:string, compilefile:string, binaryfile:string, header:string, cookie:string, follow_location:true|false, timeout:integer, sslversion:integer, request:string, username:string, password:string, accept_encoding: string]);

DESCRIPTION

This module is based on libcurl : http://curl.haxx.se/libcurl/.

PARAMETERS

url: Specifies the url to read.

error: Captures error.

accept_encoding

Specify encoding. From libcurl:

Set accept_encoding to NULL to explicitly disable it, which makes libcurl not send an Accept-Encoding: header and not decompress received contents automatically.

You can also opt to just include the Accept-Encoding: header in your request with CURLOPT_HTTPHEADER but then there will be no automatic decompressing when receiving data.

cookiefile

Specify the name for a cookie file.

From documentation of curl:

Pass the data to the HTTP server in the Cookie header. It is supposedly the data previously received from the server in a "Set-Cookie:" line. The data should be in the format "NAME1=VALUE1; NAME2=VALUE2".

If no '=' symbol is used in the argument, it is instead treated as a filename to read previously stored cookie from. This option also activates the cookie engine which will make curl record incoming cookies, which may be handy if you're using this in combination with the -L, --location option or do multiple URL transfers on the same invoke. If the file name is exactly a minus ("-"), curl will instead the contents from stdin.

The file format of the file to read cookies from should be plain HTTP headers (Set-Cookie style) or the Netscape/Mozilla cookie file format.

The file specified with -b, --cookie is only used as input. No cookies will be written to the file. To store cookies, use the -c, --cookie-jar option.

Exercise caution if you are using this option and multiple transfers may occur. If you use the NAME1=VALUE1; format, or in a file use the Set-Cookie format and don't specify a domain, then the cookie is sent for any domain (even after redirects are followed) and cannot be modified by a server-set cookie. If the cookie engine is enabled and a server sets a cookie of the same name then both will be sent on a future transfer to that server, likely not what you intended. To address these issues set a domain in Set-Cookie (doing that will include sub domains) or use the Netscape format.

If this option is used several times, the last one will be used.

Users very often want to both read cookies from a file and write updated cookies back to a file, so using both -b, --cookie and -c, --cookie-jar in the same command line is common.

data: Passes this data string to the URL. Pass a char as parameter, pointing to the full data to send in a HTTP POST operation. You must make sure that the data is formatted the way you want the server to receive it. libcurl will not convert or encode it for you in any way. For example, the web server may assume that this data is url-encoded. If you pass a JSON string as in data : { ... }, this data will be converted to a char string representing the JSON value.

writefile: The resulting body is sent to this file.

file: If the extention of the file is sn, this is treated as compilefile. If the extention is snc, then it is treated as binaryfile. Otherwise, it is treated as textfile.

textfile: Sends this file to the URL.

compilefile: Compiles, executes and sends result to the URL.

binaryfile: Executes and sends result to the URL.

header: Sends header to the URL.

cookie: Sends cookie to the URL.

follow_location: Tells the library to follow any Location: header that the server sends as part of a HTTP header.

timeout: Sets timeout for URL in seconds.

follow_location: Tells the library to follow any Location: header that the server sends as part of a HTTP header.

username: Specify a username. sets the user name to be used in protocol authentication

password: Specify a password.

request: Specifies a custom request method to use. See CURLOPT_CUSTOMREQUEST for more information.

sslversion: Specifies a specific ssl version to use. See CURLOPT_SSLVERSION for more information.

httppost: This parameter is used to send data is multipart/form-data format. This field is expecting an array of name, value. if the value is an array, it will consider the value to be a filename which will be send as-is.; This parameter is available from version 2.4 of this module.

RETURN

A context with cookies, headers and body. Also, if the data is sent to the URL, the last post is set in variable post.

Starting from version 1.5 status code is also return.

MODULES

To use this module, one must specify the following in /usr/local/website/site.conf:

Module geoip {
    Library : "/usr/local/lib/libsncurl.so"
    Init : "sncurl"
}

EXAMPLES

Note: In the followings examples, the _ between the { should be removed to make it work.

        res = post(
                    request: "POST",
                    url: URL_ADD_CLINIC,
                    header : "Authorization: Basic " .+ base64_encode(USERNAME .+ ":" .+ PASSWORD),
                    error:t,
                    data: [
                        { "uuid_clinic" : "uuid"},
                        { "clinic_name" : "nom de la clinique"},
                        { "logo" : [ "/html/images/logo-sednove.png"]}
                    ]
                );

res=<{curl(error: t,xx:"50.21.175.111"); t.errcode;}>. return .*Parameter xx is not valid in curl.*

res=<{r=curl(error: t,url:"http://v5.sednove.ca/curl1.sn"); t.errcode; r.body}>.),
res=0Test 1 de curl
.

res=<{r=curl(error: t,url:"http://v5.sednove.ca/curl2.sn"); t.errcode; r.cookies;
r=curl(error: t,cookie:r.cookies,url:"http://v5.sednove.ca/curl2.sn"); t.errcode; r.body;
}>.),
res=0sednove=pierre; 0Test 2 de curl cgidata={}cookies=pierre
.

res=<{r=curl(error: t,data:"x=yéé",url:"http://v5.sednove.ca/curl3.sn"); t.errcode; r.body}>.),
res=0Test 3 de curl cgidata={"x":"yéé"}.

res=<{curl(error: t,data:"x=yéé",url:"http://v5.sednove.ca/curl5.sn",timeout:2); t;}>.),
res=.*The requested URL /curl5.sn was not found.*

res=<{r=curl(error: t,data:"x=yéé",url:"http://v5.sednove.ca/curl4.sn",timeout:2); t.errcode; t.errmsg;}>.),
res=14At line=1,errcode=14 Curl error: a timeout was reached..

res=<{
        remove("/tmp/curl6.txt");
        r=curl(error: t,data:"x=yéé",url:"http://v5.sednove.ca/curl6.sn",writefile:"/tmp/curl6.txt"); t.errcode; read("/tmp/curl6.txt");}>.),
res=0This is curl 6 {"x":"yéé"}..

res=<{
        r=curl(error: t,data:"x=yéé",follow_location:false,url:"http://v5.sednove.ca/curl7.sn"); t.errcode; r.body;}>.),
.*The document has moved.*

res=<{
        r=curl(error: t,data:"x=yéé",follow_location:true,url:"http://v5.sednove.ca/curl7.sn"); t.errcode; r.body;}>.),
res=0This is curl 6 {"parm":"yéé"}..

res=<{
        r=curl(error: t,textfile:"/html/curl8.sn", url:"http://v5.sednove.ca/curl9.sn"); t.errcode; r.body;}>.),
res=0curl9
cgidata={"sn_value":"This is a text file
<{ 56 \* 65, }>
"}.

res=<{
        r=curl(error: t,file:"/html/curl8.sn", url:"http://v5.sednove.ca/curl9.sn"); t.errcode; r.body;}>.),
q(res=0curl9
cgidata={"sn_value":"This is a text file
3640
"}.

res=<{
        remove("/html/curl.snc");
        compile(src:"/html/curl8.sn",dst:"/html/curl8.snc");
        r=curl(error: t,file:"/html/curl8.snc", url:"http://v5.sednove.ca/curl9.sn"); t.errcode; r.body;}>.),
res=0curl9
cgidata={"sn_value":"This is a text file
3640
"}.

curl files:

curl1:
Test 1 de curl

curl2:
Test 2 de curl <{
a = cgidata();
"cgidata="; a;
a = cookies();
"cookies="; a.sednove;
cookies(name:"sednove",value:"pierre",path:"/");
}>

curl3:
Test 3 de curl <{
a = cgidata();
"cgidata="; a;
}>

curl4:
Test 4 de curl <{
sleep(10);
}>

curl6:
This is curl 6 <{ a=cgidata(); a; }>.

curl7:
<{a=cgidata(); redirect("http://v5.sednove.ca/curl6.sn?parm=" .+ a.x); }>
<html>
        <head>
                <meta http-equiv="refresh" content="5; url=http://www.sednove.com/">
        </head>
        <body>
                This is a test
        </body>
</html>

curl8:
This is a text file
<{ 56 * 65; }>

curl9:
curl9
<{
        a = cgidata();
        "cgidata="; a;
}>

curl 20:
This sets a paypal billing plan to ACTIVE using PATCH method and TSL v1.2
{_{      
    res = curl(url: "https://api.sandbox.paypal.com/v1/payments/billing-plans/" .+ plan_id ,
                header: "Content-Type: application/json",
                header: "Authorization: Bearer <Access Token>",
                data: '[{ "path": "/",  "value": { "state": "ACTIVE" }, "op": "replace" }]',
                customrequest:"PATCH", //updates only certain fields
                sslversion:6 //uses TLS v1.2
            );
}_}

AUTHOR

Written by Pierre Laplante and Caroline Laplante, <laplante@sednove.com>

MODIFICATIONS

1.0 2014-09-09 21:24:14 laplante@sednove.com

1.1 2014-11-12 laplante@sednove.com Add verifypeer arguement

1.2 2018-10-02 laplante@sednove.com Add parameter cookiefile

Edit