Merges annotations stored in a JSON file into the data.
Plugin: built-in
data The data to be annotated.
key The primary key used for joining the annotated data.
path Path to the JSON file containing the annotations.
annotations The imported annotations only.
data The input data with annotations.
Removes duplicates from a collection using a unique primary key attribute
Plugin: built-in
attribute No description.
data No description.
pick No description.
data No description.
removed No description.
Downloads files.
Plugin: built-in
attribute Target attribute where the path to the downloaded files are written to. Will be an array if "map" creates an array. (default: "files")
batchSize Number of files to download in parallel. (default: 5)
clean Remove files in target folder that weren't scheduled for download.
data The source data that will be mapped.
map A function that maps the data item to an object containing the "name" (optional, new filename for the downloaded file) and "url" (required, URL of the file to download). Can also map to an array, in which case multiple files are downloaded for each data item.
options Options for got.
postprocess Run a process per download, with the file path as the first argument.
skip If true, skips downloading if the file already exists.
target The target path to store the files at. (default: ".")
data No description.
paths No description.
Applies a filter function to a collection.
Plugin: built-in
data The data to filter.
filter One or more filter functions that will be applied to each data item.
data No description.
filtered No description.
Filters a collection my matching attribute values against a set.
Plugin: built-in
attribute No description.
data No description.
values No description.
data No description.Filters a collection by requiring values for one or more attributes to be in a specified range.
Plugin: built-in
data No description.data No description.Filters a collection by defining a list of included rows by index.
Plugin: built-in
data No description.data No description.Joins two collections via a primary key.
Plugin: built-in
affluent The collection to merge into the source data.
annotate Merge additional data into successfully joined items.
attribute If defined, put the joined data into a new attribute in the source data.
data The source data that is joined into.
key One or more keys to join the two collections with.
preserve If true, affluent keys with the same value preserve the original data key.
data The joined data.
matched Contains only the matched data.
unmatched Data (not affluent data!) that was not matched during the join.
Converts items in a collection using a mapping function.
Plugin: built-in
data The data to filter.
map One or more mapping functions that will be applied to each data item.
data No description.Matches, modifies or creates attributes by extracting capture groups from regular expressions.
Plugin: built-in
data The data to modify.
match Maps a data attribute to one or more regular expressions. If the regular expression contains capture groups, each expression will assign their captured values back to the attribute or an attribute targeted by the name of the respective capture group.
data The modified data.
matched The modified data, but filtered to contain only items that were matched.
matches Shows all matches -- for debugging.
unmatched The modified data, but filtered to contain only items that were not matched.
Pipes an entire collection into a terminal command via stdin, and reads the result back from stdout.
Plugin: built-in
command The shell command to execute.
data The data that is piped into the command via stdin.
deserialise A function that deserialises the data that is read back from stdout. If unspecified, it will be a string.
serialise A function that serialises the data that is piped into the process via stdin. If unspecified, toString() will be called on the data object.
data The data read back from stdout.Reads JSON data.
Plugin: built-in
options No description.
uri No description.
data No description.Reads YAML data.
Plugin: built-in
options No description.
uri No description.
data No description.Runs a terminal command, optionally for each data item.
Plugin: built-in
command A terminal command, or a callback that takes a data item and returns the command to execute for the item.
data If data is supplied, run the command for each item. The "command" port has to be a callback function returning the command to execute, otherwise the command is only run once.
stdio If unset, the output is written into the stdout port. This configuration allows configuration of the output pipe via https://nodejs.org/api/child_process.html#child_process_options_stdio
data The original data.
stdout A list of command outputs (standard out).
Sorts data.
Plugin: built-in
data No description.
orderBy No description.
orders No description.
data No description.
unsortable No description.
Writes a collection to a JSON file.
Plugin: built-in
attributes Only serialise the listed attributes.
data No description.
path No description. (default: "data.json")
pretty No description.
stable No description.
None