splitx
Split a file into pieces.
Mirrors GNU split. Run it as splitx, or as gnux splitx.
Synopsis
splitx [OPTION]... [FILE [PREFIX]]GNU reference
Description
splitx copies FILE (standard input when FILE is - or absent) into files named PREFIX (x by default) plus a suffix: xaa, xab, and so on, 1000 lines per file. -l N puts N lines in each file (the obsolete -N form too), -b SIZE puts SIZE bytes, and -C SIZE as many whole lines as fit in SIZE bytes, splitting only a line longer than that. SIZE takes the suffixes K, M, G, ... (powers of 1024) and KB, MB, ... (powers of 1000). -n CHUNKS makes a fixed number of files instead: N files of nearly equal size (the first ones a byte longer when the size does not divide), l/N without splitting lines (each line goes to the chunk it starts in), or r/N dealing lines out in turn; K/N, l/K/N and r/K/N write only the Kth chunk to standard output. To size the chunks of a pipe, splitx first copies it to a temporary file, as GNU does. Suffixes are two letters, and when they run out they grow (xyz is followed by xzaaa) so the names still sort in order; -a N fixes their length, -d and -x make them decimal or hexadecimal, --numeric-suffixes=FROM and --hex-suffixes=FROM start them at FROM, and --additional-suffix appends a fixed string such as .csv. -t SEP splits lines on SEP instead of newline ('\0' for NUL), -e skips empty files in the -n modes, and --verbose reports each file as it is created. splitx refuses to overwrite FILE with one of its pieces. Only one way of splitting may be given. GNU's --filter, which pipes each piece through a shell command, is not implemented; -u is accepted and has no effect, since output is never held back. The digits of two obsolete -N options in a row run together (-5 -3 reads as -53).
Permissions
Needs read permission on FILE, and write permission on the directory the pieces are created in (and on any existing piece, which is overwritten).
Use cases
- Breaking a file too large for an email attachment or a FAT32 drive into pieces.
- Dividing a big CSV or log into batches for parallel processing.
- Spreading work over N workers with
-nr/N or-nl/N. - Sampling one slice of a large file with
-nK/N without writing the rest. - Reassembling the pieces later with
catx, since the names sort in order.
Flags
--additional-suffix SUFFIX append an additional SUFFIX to file names
-b, --bytes SIZE put SIZE bytes per output file
-e, --elide-empty-files do not generate empty output files with '-n'
--help display this help and exit
-x, --hex-suffixes FROM[= ]�use hex suffixes starting at 0, not alphabetic; --hex-suffixes=FROM starts at FROM
-C, --line-bytes SIZE put at most SIZE bytes of records per output file
-l, --lines NUMBER put NUMBER lines/records per output file
-n, --number CHUNKS generate CHUNKS output files: N, K/N, l/N, l/K/N, r/N or r/K/N
-d, --numeric-suffixes FROM[= ]�use numeric suffixes starting at 0, not alphabetic; --numeric-suffixes=FROM starts at FROM
-t, --separator SEP use SEP instead of newline as the record separator; '\0' (zero) specifies the NUL character
-a, --suffix-length N generate suffixes of length N (default 2)
-u, --unbuffered immediately copy input to output with '-n r/...' (always the case here)
--verbose print a diagnostic just before each output file is opened
--version output version information and exitExamples
Cut a 10 GB backup into 2 GB pieces that fit on a FAT32 USB drive
splitx -b 2G backup.tar.gz backup.tar.gz.part-(creates backup.tar.gz.part-aa ... backup.tar.gz.part-ae)Put the pieces back together on the other machine
catx backup.tar.gz.part-* > backup.tar.gzBreak a million-row CSV into files of 100,000 rows for an import tool that chokes on big files
tailx -n +2 users.csv | splitx -l 100000 -d --additional-suffix=.csv - users-(creates users-00.csv ... users-09.csv)Split a log into 4 roughly equal parts for 4 parallel workers, without cutting a line in half
splitx -n l/4 access.log chunk-Deal a list of URLs out to 3 crawlers in turn
splitx -n r/3 urls.txt crawler-Look at just the third quarter of a huge file without writing any pieces
splitx -n l/3/4 huge.log | headxSend a log in email-sized pieces of at most 5 MB, keeping lines whole
splitx -C 5M app.log app.log.Number the pieces from 1 instead of 0
splitx -l 500 --numeric-suffixes=1 -a 3 list.txt list-(creates list-001, list-002, ...)See each file as it is created
splitx --verbose -l 2 names.txtcreating file 'xaa'
creating file 'xab'Split NUL-separated records, such as the output of find -print0
find . -type f -print0 | splitx -t '\0' -l 1000 - batch-Split stdin into 1,000-line files the obsolete way, as old scripts do
seqx 2500 | splitx -1000Split and reassemble in the portable gnux shell, the same on every OS
gnux shell -c 'splitx -b 1M video.mp4 part- && catx part-* > copy.mp4'Options
Each option has a page of its own, with its usage and examples.
- --additional-suffix: Append an additional SUFFIX to file names.
- -b, --bytes: Put SIZE bytes per output file.
- -e, --elide-empty-files: Do not generate empty output files with '
-n'. - -x, --hex-suffixes: Use hex suffixes starting at 0, not alphabetic;
--hex-suffixes=FROM starts at FROM. - -C, --line-bytes: Put at most SIZE bytes of records per output file.
- -l, --lines: Put NUMBER lines/records per output file.
- -n, --number: Generate CHUNKS output files: N, K/N, l/N, l/K/N, r/N or r/K/N.
- -d, --numeric-suffixes: Use numeric suffixes starting at 0, not alphabetic;
--numeric-suffixes=FROM starts at FROM. - -t, --separator: Use SEP instead of newline as the record separator; '\0' (zero) specifies the NUL character.
- -a, --suffix-length: Generate suffixes of length N (default 2).
- -u, --unbuffered: Immediately copy input to output with '
-nr/...' (always the case here). - --verbose: Print a diagnostic just before each output file is opened.
Exit status
| Status | Meaning |
|---|---|
| 0 | success: every operand was processed |
| 1 | failure: an operand could not be processed (diagnosed on standard error), or a command-line mistake |
Help in the terminal
splitx --help # options, first examples, exit status
gnux help splitx # this page: the full manual
gnux help splitx --OPTION # one option in depth, with its examples