curl with partial files | daniel.haxx.se

Back in September 2023, we extended the curl command-line tool with a fairly advanced and flexible new tool variable system. Using it, users can use archives, environment variables, and more—almost all existing command line options—in powerful ways when building curl command lines in ways that were previously impossible.

The curl command line was already very powerful before, but these new variables definitely take it up a few notches.

Coming February 2025

In the upcoming curl 8.12.0 version, we have further expanded this variable support. From now on, you can assign a variable to hold the contents of a partial document. Gets a range of bytes in a given archive into a variable and uses that variable in the command line instead of using the entire archive.

You can get the first few bytes and use it as a username, you can get the hundred bytes in the middle of the archive and publish it or do countless other things.

Byte range

You are asking curl to read a range of bytes from the file instead of reading the entire range of bytes via append [n-M] When you assign a variable, add it to the variable name. Where N and M are the offsets of the first and last bytes in the file, and 0 is the first byte. If the second number is omitted, it means until the end of the file.

For example, get the first 32 bytes from a file named secret and set as password Daniel:

curl --variable "pwd[0-31]@secret" \
--expand-user daniel:{{pwd}} \
https://example.com/

Skip the first thousand bytes in the file named local file And send the rest via POST:

curl --variable "upload[1000-]@localfile" \
--expand-post '{{upload}}' \
https://example.com/

Have function

You can of course also combine the byte offset with the standard Extended functions. For example, get the first one hundred bytes from a file named random And send them Base64 encoding as POST:

curl --variable "binary[0-99]@random" \
--expand-post '{{binary:b64}}' \
https://example.com/

I hope you will like it.

renew

After his article was first published, we discussed the exact syntax of this feature and decided to tweak it a bit to reduce the chance of older curl versions being tricked when trying new command-line options.

This version now displays updated syntax.