We recently ran into a problem with a large array. The number of items in the array was too large to process them at once. We wanted to offload smaller chunks of these items to a queue worker.
This tutorial shows you how to split an array into a list of smaller chunks.
Node.js Series Overview
- String Replace All Appearances
- Remove All Whitespace From a String in JavaScript
- Generate a Random String in Node.js or JavaScript
- Remove Extra Spaces From a String in JavaScript or Node.js
- Remove Numbers From a String in JavaScript or Node.js
- Get the Part Before a Character in a String in JavaScript or Node.js
- Get the Part After a Character in a String in JavaScript or Node.js
- How to Check if a Value is a String in JavaScript or Node.js
- Check If a String Includes All Strings in JavaScript/Node.js/TypeScript
- Check if a Value is a String in JavaScript and Node.js
- Limit and Truncate a String to a Given Length in JavaScript and Node.js
- How to Run an Asynchronous Function in Array.map()
- How to Reset and Empty an Array
- for…of vs. for…in Loops
- Clone/Copy an Array in JavaScript and Node.js
- Get an Array With Unique Values (Delete Duplicates)
- Sort an Array of Integers in JavaScript and Node.js
- Sort a Boolean Array in JavaScript, TypeScript, or Node.js
- Check If an Array Contains a Given Value in JavaScript or Node.js
- Add an Item to the Beginning of an Array in JavaScript or Node.js
- Append an Item at the End of an Array in JavaScript or Node.js
- How to Exit and Stop a for Loop in JavaScript and Node.js (Coming soon)
- Split an Array Into Smaller Array Chunks in JavaScript and Node.js
- How to Get an Index in a for…of Loop in JavaScript and Node.js (Coming soon)
- How to Exit, Stop, or Break an Array#forEach Loop in JavaScript or Node.js (Coming soon)
- Callback and Promise Support in your Node.js Modules
- Run Async Functions/Promises in Sequence
- Run Async Functions/Promises in Parallel
- Run Async Functions in Batches
- How to Fix “Promise resolver undefined is not a function” in Node.js or JavaScript
- Detect if Value Is a Promise in Node.js and JavaScript
- Get a File’s Created Date
- Get a File’s Last Modified/Updated Date
- How to Create an Empty File
- Check If a Path or File Exists
- How to Rename a File
- Check If a Path Is a Directory (Coming soon)
- Check If a Path Is a File (Coming soon)
- Retrieve the Path to the User’s Home Directory (Coming soon)
- How to Touch a File (Coming soon)
Divide and Chunk an Array Into Smaller Arrays
This approach of dividing an array into smaller chunks assumes a fixed chunk size. For example, we want to split the array [1, 2, 3, 4, 5, 6, 7, 8]
into chunks of three items. The result of chunking the source array should look like this:
[
[1, 2, 3],
[4, 5, 6],
[7, 8]
]
JavaScript comes with the Array#splice
method. The splice
method removes items from an array and returns them as a new array. This way, you can remove the number of items from the source array until it’s empty.
Here’s a sample implementation for a chunk(items, size)
function:
/**
* Split the `items` array into multiple, smaller arrays of the given `size`.
*
* @param {Array} items
* @param {Number} size
*
* @returns {Array[]}
*/
function chunk (items, size) {
const chunks = []
items = [].concat(...items)
while (items.length) {
chunks.push(
items.splice(0, size)
)
}
return chunks
}
// Example
const items = [1, 2, 3, 4, 5, 6, 7, 8]
chunk(items, 3)
// [ [1, 2, 3], [4, 5, 6], [7, 8] ]
The basic idea of this function is to remove items from the source array and push the removed elements array to the result array. The result contains all smaller chunks for further processing.
Use the @supercharge/collections Package
I’m the maintainer of the @supercharge/collections package providing convenient array utilities. This package comes with the .chunk(<size>)
method splitting the given array into arrays of the given size
.
Wrapping an array using the @supercharge/collections
package creates a collection instance. You must end the chain with .all()
to retrieve the underlying JavaScript array:
const Collect = require('@supercharge/collections')
Collect([1, 2, 3, 4, 5, 6, 7, 8]).chunk(3).all()
// [ [1, 2, 3], [4, 5, 6], [7, 8] ]
Collect([1, 2, 3, 4, 5, 6, 7, 8]).chunk(15).all()
// [ [1, 2, 3, 4, 5, 6, 7, 8] ]
Enjoy!
Mentioned Resources
- Array#splice docs in the MDN (Mozilla Developer Network)
- @supercharge/collections repository on GitHub