File size: 9,638 Bytes
369fac9
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
# get-stream

> Get a stream as a string, Buffer, ArrayBuffer or array

## Features

- Works in any JavaScript environment ([Node.js](#nodejs-streams), [browsers](#web-streams), etc.).
- Supports [text streams](#getstreamstream-options), [binary streams](#getstreamasbufferstream-options) and [object streams](#getstreamasarraystream-options).
- Supports [async iterables](#async-iterables).
- Can set a [maximum stream size](#maxbuffer).
- Returns [partially read data](#errors) when the stream errors.
- [Fast](#benchmarks).

## Install

```sh
npm install get-stream
```

## Usage

### Node.js streams

```js
import fs from 'node:fs';
import getStream from 'get-stream';

const stream = fs.createReadStream('unicorn.txt');

console.log(await getStream(stream));
/*
              ,,))))))));,
           __)))))))))))))),
\|/       -\(((((''''((((((((.
-*-==//////((''  .     `)))))),
/|\      ))| o    ;-.    '(((((                                  ,(,
         ( `|    /  )    ;))))'                               ,_))^;(~
            |   |   |   ,))((((_     _____------~~~-.        %,;(;(>';'~
            o_);   ;    )))(((` ~---~  `::           \      %%~~)(v;(`('~
                  ;    ''''````         `:       `:::|\,__,%%    );`'; ~
                 |   _                )     /      `:|`----'     `-'
           ______/\/~    |                 /        /
         /~;;.____/;;'  /          ___--,-(   `;;;/
        / //  _;______;'------~~~~~    /;;/\    /
       //  | |                        / ;   \;;,\
      (<_  | ;                      /',/-----'  _>
       \_| ||_                     //~;~~~~~~~~~
           `\_|                   (,~~
                                   \~\
                                    ~~
*/
```

### Web streams

```js
import getStream from 'get-stream';

const {body: readableStream} = await fetch('https://example.com');
console.log(await getStream(readableStream));
```

### Async iterables

```js
import {opendir} from 'node:fs/promises';
import {getStreamAsArray} from 'get-stream';

const asyncIterable = await opendir(directory);
console.log(await getStreamAsArray(asyncIterable));
```

## API

The following methods read the stream's contents and return it as a promise.

### getStream(stream, options?)

`stream`: [`stream.Readable`](https://nodejs.org/api/stream.html#class-streamreadable), [`ReadableStream`](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream), or [`AsyncIterable<string | Buffer | ArrayBuffer | DataView | TypedArray>`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols#the_async_iterator_and_async_iterable_protocols)\
`options`: [`Options`](#options)

Get the given `stream` as a string.

### getStreamAsBuffer(stream, options?)

Get the given `stream` as a Node.js [`Buffer`](https://nodejs.org/api/buffer.html#class-buffer).

```js
import {getStreamAsBuffer} from 'get-stream';

const stream = fs.createReadStream('unicorn.png');
console.log(await getStreamAsBuffer(stream));
```

### getStreamAsArrayBuffer(stream, options?)

Get the given `stream` as an [`ArrayBuffer`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer).

```js
import {getStreamAsArrayBuffer} from 'get-stream';

const {body: readableStream} = await fetch('https://example.com');
console.log(await getStreamAsArrayBuffer(readableStream));
```

### getStreamAsArray(stream, options?)

Get the given `stream` as an array. Unlike [other methods](#api), this supports [streams of objects](https://nodejs.org/api/stream.html#object-mode).

```js
import {getStreamAsArray} from 'get-stream';

const {body: readableStream} = await fetch('https://example.com');
console.log(await getStreamAsArray(readableStream));
```

#### options

Type: `object`

##### maxBuffer

Type: `number`\
Default: `Infinity`

Maximum length of the stream. If exceeded, the promise will be rejected with a `MaxBufferError`.

Depending on the [method](#api), the length is measured with [`string.length`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/length), [`buffer.length`](https://nodejs.org/api/buffer.html#buflength), [`arrayBuffer.byteLength`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer/byteLength) or [`array.length`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/length).

## Errors

If the stream errors, the returned promise will be rejected with the `error`. Any contents already read from the stream will be set to `error.bufferedData`, which is a `string`, a `Buffer`, an `ArrayBuffer` or an array depending on the [method used](#api).

```js
import getStream from 'get-stream';

try {
	await getStream(streamThatErrorsAtTheEnd('unicorn'));
} catch (error) {
	console.log(error.bufferedData);
	//=> 'unicorn'
}
```

## Tips

### Alternatives

If you do not need the [`maxBuffer`](#maxbuffer) option, [`error.bufferedData`](#errors), nor browser support, you can use the following methods instead of this package.

#### [`streamConsumers.text()`](https://nodejs.org/api/webstreams.html#streamconsumerstextstream)

```js
import fs from 'node:fs';
import {text} from 'node:stream/consumers';

const stream = fs.createReadStream('unicorn.txt', {encoding: 'utf8'});
console.log(await text(stream))
```

#### [`streamConsumers.buffer()`](https://nodejs.org/api/webstreams.html#streamconsumersbufferstream)

```js
import {buffer} from 'node:stream/consumers';

console.log(await buffer(stream))
```

#### [`streamConsumers.arrayBuffer()`](https://nodejs.org/api/webstreams.html#streamconsumersarraybufferstream)

```js
import {arrayBuffer} from 'node:stream/consumers';

console.log(await arrayBuffer(stream))
```

#### [`readable.toArray()`](https://nodejs.org/api/stream.html#readabletoarrayoptions)

```js
console.log(await stream.toArray())
```

#### [`Array.fromAsync()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/fromAsync)

If your [environment supports it](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/fromAsync#browser_compatibility):

```js
console.log(await Array.fromAsync(stream))
```

### Non-UTF-8 encoding

When all of the following conditions apply:
  - [`getStream()`](#getstreamstream-options) is used (as opposed to [`getStreamAsBuffer()`](#getstreamasbufferstream-options) or [`getStreamAsArrayBuffer()`](#getstreamasarraybufferstream-options))
  - The stream is binary (not text)
  - The stream's encoding is not UTF-8 (for example, it is UTF-16, hexadecimal, or Base64)

Then the stream must be decoded using a transform stream like [`TextDecoderStream`](https://developer.mozilla.org/en-US/docs/Web/API/TextDecoderStream) or [`b64`](https://github.com/hapijs/b64).

```js
import getStream from 'get-stream';

const textDecoderStream = new TextDecoderStream('utf-16le');
const {body: readableStream} = await fetch('https://example.com');
console.log(await getStream(readableStream.pipeThrough(textDecoderStream)));
```

### Blobs

[`getStreamAsArrayBuffer()`](#getstreamasarraybufferstream-options) can be used to create [Blobs](https://developer.mozilla.org/en-US/docs/Web/API/Blob).

```js
import {getStreamAsArrayBuffer} from 'get-stream';

const stream = fs.createReadStream('unicorn.txt');
console.log(new Blob([await getStreamAsArrayBuffer(stream)]));
```

### JSON streaming

[`getStreamAsArray()`](#getstreamasarraystream-options) can be combined with JSON streaming utilities to parse JSON incrementally.

```js
import fs from 'node:fs';
import {compose as composeStreams} from 'node:stream';
import {getStreamAsArray} from 'get-stream';
import streamJson from 'stream-json';
import streamJsonArray from 'stream-json/streamers/StreamArray.js';

const stream = fs.createReadStream('big-array-of-objects.json');
console.log(await getStreamAsArray(
	composeStreams(stream, streamJson.parser(), streamJsonArray.streamArray()),
));
```

## Benchmarks

### Node.js stream (100 MB, binary)

- `getStream()`: 142ms
- `text()`: 139ms
- `getStreamAsBuffer()`: 106ms
- `buffer()`: 83ms
- `getStreamAsArrayBuffer()`: 105ms
- `arrayBuffer()`: 81ms
- `getStreamAsArray()`: 24ms
- `stream.toArray()`: 21ms

### Node.js stream (100 MB, text)

- `getStream()`: 90ms
- `text()`: 89ms
- `getStreamAsBuffer()`: 127ms
- `buffer()`: 192ms
- `getStreamAsArrayBuffer()`: 129ms
- `arrayBuffer()`: 195ms
- `getStreamAsArray()`: 89ms
- `stream.toArray()`: 90ms

### Web ReadableStream (100 MB, binary)

- `getStream()`: 223ms
- `text()`: 221ms
- `getStreamAsBuffer()`: 182ms
- `buffer()`: 153ms
- `getStreamAsArrayBuffer()`: 171ms
- `arrayBuffer()`: 155ms
- `getStreamAsArray()`: 83ms

### Web ReadableStream (100 MB, text)

- `getStream()`: 141ms
- `text()`: 139ms
- `getStreamAsBuffer()`: 91ms
- `buffer()`: 80ms
- `getStreamAsArrayBuffer()`: 89ms
- `arrayBuffer()`: 81ms
- `getStreamAsArray()`: 21ms

[Benchmarks' source file](benchmarks/index.js).

## FAQ

### How is this different from [`concat-stream`](https://github.com/maxogden/concat-stream)?

This module accepts a stream instead of being one and returns a promise instead of using a callback. The API is simpler and it only supports returning a string, `Buffer`, an `ArrayBuffer` or an array. It doesn't have a fragile type inference. You explicitly choose what you want. And it doesn't depend on the huge `readable-stream` package.

## Related

- [get-stdin](https://github.com/sindresorhus/get-stdin) - Get stdin as a string or buffer
- [into-stream](https://github.com/sindresorhus/into-stream) - The opposite of this package