Hacker Read

frjalex · 2019-03-28 02:07:04

libcurl is open source, I would imagine it'd not be difficult to make it "lie."

cp9 | karma 809 | avg karma 5.36 · | 2022-12-30 17:17:48

I’d suggest learning what libcurl is used for before throwing it out with the bath water ¯\_(?)_/¯

nommm-nommm | karma 4083 | avg karma 2.09 · | 2016-01-20 00:29:32

That's all waaaaayyyy way too much (and too technical) information to throw at someone. Nobody cares what libcurl is or what it does or if its given away for free or not. Keep it simple and to the point. 1) Sorry I can't help you and 2) here's who can. Thats it.

pimlottc | karma 7866 | avg karma 3.44 · | 2024-01-29 10:49:45

Someone should buy libcurl.com and make it return the source code to generate a request to itself. Bonus points for setting the same headers and options as the triggering request.

tlrobinson | karma 30498 | avg karma 4.18 · | 2015-03-20 18:49:21+00:00

As someone else mentioned, this is actually built into Curl with the "--libcurl file.c" option.

leosanchez | karma 342 | avg karma 1.64 · | 2024-03-09 20:03:23

You can use hurl which uses libcurl. You can save it to git and commit your hurl testa in the same repo.

bagder | karma 836 | avg karma 19.0 · | 2015-03-20 09:20:15+00:00

curl has a --libcurl option which you can add to a curl command line, and then it'll generate a libcurl-using C code template for that same operation...

m0d0nne11 | karma 47 | avg karma 0.72 · | 2024-01-29 15:09:37

SEE ALSO: curl --libcurl

mschuster91 | karma 19531 | avg karma 1.6 · | 2023-10-05 13:11:03

A lot of stuff depends on cURL/libcurl. IIRC, php these days has it enabled by default.

colonwqbang | karma 2093 | avg karma 3.76 · | 2021-05-02 08:50:55+00:00

Libcurl is the only sane way I know to access the web from a C or C++ environment. I expect that most C programs needing to do web requests will link in libcurl.

brianwski | karma 1856 | avg karma 4.62 · | 2020-01-15 07:22:53+00:00

Backblaze CTO here....

> I found libcurl difficult to use in 'C'

Umm.... it is literally the most simple API that could possibly exist? Here are the four lines in 'C' to fetch a URL:

curl_global_init(CURL_GLOBAL_SSL); // do this once inside of "main()"

CURL ch = curl_easy_init();

curl_easy_setopt(ch, CURLOPT_URL, "http://www.google.com");

int res = curl_easy_perform(ch);

I'm not sure how you could fetch a URL without initializing the library or specifying the URL?? It is like profoundly the most simple API that could possibly exist, I challenge you to specify any imagined API that is shorter or more simple or more straight-forward??!

The nice thing about the libcurl API is the above works and is the "base" concept, but if you want to specify something like a "network throttle" to be polite to other things on the network, you just start adding one extra line at a time to the above code:

curl_easy_setopt(ch, CURLOPT_MAX_SEND_SPEED_LARGE, (curl_off_t) (256 * 1024)); // 256 kbits/sec

reply

carllerche | karma 1590 | avg karma 14.45 · | 2014-12-17 21:39:20

Well, http://github.com/carllerche/curl-rust has been working great for me. Sure, it uses libcurl so it isn't pure Rust, but that doesn't make it not work :-)

bombolo | karma 934 | avg karma 0.72 · | 2023-03-01 06:04:04

I once saw him at a conference and he said curl even has an option to emit the C code to call libcurl and do the curl request you did.

I wish I had known this before.

However I mostly use wget from shell, and since I mostly work with python aiohttp/standard library/just writing the request on the socket.

reply

cbsmith | karma 4783 | avg karma 1.38 · | 2023-08-27 12:54:19

Presumably so. You'd just need to compile the curl-impersonate for the appropriate browser you want to impersonate, and then link it in to a proxy that used libcurl for its https traffic.

agildehaus | karma 2471 | avg karma 3.72 · | 2014-08-30 20:49:55

How would it be? It's just using cURL to send a file to a URL.

nicoster | karma 9 | avg karma 0.82 · | 2024-01-31 23:46:29

this may not be as useful as it appears since different requests are for different different URLs, unless you redirect a request to libcurl.com

capableweb | karma 37790 | avg karma 4.15 · | 2022-04-06 03:34:26

libcurl wrapper - https://github.com/sepisoad/jurl

json encoder/decoder - https://github.com/janet-lang/json

Not aware of any HTML parsers/traverses (if that's what you mean with "Crawler") but interfacing with C libraries is easy so grab any C library you'd like for that task.

reply

iudqnolq | karma 5689 | avg karma 2.13 · | 2019-11-24 05:53:07

Since there's no interactivity or logins or anything complex curl would probably do the job.

jicea | karma 1453 | avg karma 4.94 · | 2022-05-10 23:51:25

As a maintainer, I feel obligated to mention Hurl [1], a tool for testing HTTP with plain text and curl. It’s a wrapper around libcurl, in a single binary, with syntactic sugar for asserts. You can use it also like curl to give you an HTTP output.

[1] https://hurl.dev

reply

thejosh | karma 5634 | avg karma 3.96 · | 2016-11-02 10:20:57+00:00

Apart from the oodles of software that depends on curl/libcurl, my favourite thing is to "Copy as cURL" from a Chrome request.