Buffer acts as a temporary storage for a chunk of data that is to be transferred from one place to another.

What buffers Actually does in Node js

First of all, the buffer is filled with data, then after that, it is being passed along.Buffer collects small Chunks of data at a time.Instead of waiting for all of the data to be stored in memory what we do we transfer small chunks of data in a buffer and buffer collects small chunks of data and when the buffer is full, we pass that chunk of data down the stream it is processed and sent to the client.

Topics Covered

    • Why we use Buffer?
    • How Does Buffer Looks Like?
  • Ways of creating buffers using Buffer.alloc(), Buffer.allocUnsafe()
    • Buffer.alloc()
    • Buffer.allocUnsafe()
  • Buffer Class
    • Buffer from a given array
    • Buffer from a given string
  • Buffer Operations
    • Writing to Buffer
    • Reading from Buffer
    • Convert Buffer to JSON
    • Concatenate Buffers
    • Compare Buffers
    • Copy Buffer
    • Slice Buffer
    • Buffer Length
    • Buffer fill
    • Buffer Index
    • Buffer Equals
Why we use Buffer?

Pure javascript works well with Unicode-encoded strings.In fact, it is the Unicode in your browser that states that 76 should represent L.Pure Javascript do not handle straight binary data very well, this is fine in browsers where most data is in the form of strings. However, Node.js have to face with reading and writing to the filesystem and TCP streams thus it makes it necessary to deal with purely binary streams of data.

Node has a way to handle binary data, Buffer class is the primary data structure in a node and used with most I/O operations.In node, each buffer represents to some raw memory allocated outside V8. A buffer acts like an array of integers, once allocated it cannot be resized.

Buffers act somewhat like arrays of integers, but aren’t resizable and have a whole bunch of methods specifically for binary data. The “integers” in buffer represents a byte and they are limited to values from 0 to 255 (2^8-1).

How does Buffer look like?

A buffer act as a container for raw bytes, byte means 8 bits, and a bit is just a 0 or 1, So byte will look like 10101. At the lowest level, all data in computer act as bits.

In node js, things like numbers, strings, booleans we work on them, and their abstraction they are based on 0’s and 1’s (bits!), thus whenever your program wants to communicate outside of Node.js we can’t  rely on those data types anymore.Therefore Communication happens in bits.

Buffer looks like this.

<Buffer 02 04 06 08 0a 0c 0e 10>

Here how can two numbers represents a byte, here the answer is in order to save space and to make it more readable node js have chosen to display a hexadecimal number instead of a binary number.

Ways of creating buffers using two methods Buffer.alloc(), Buffer.allocUnsafe().

Note: Buffer object is a global object in Node.js, therefore it is not necessary to import it using the require keyword.

Ways of creating buffers using Buffer.alloc(), Buffer.allocUnsafe() 
Buffer.alloc() 

Syntax: The Syntax of  Buffer.alloc() is given below.

  Buffer.alloc(size, fill, encoding);  

Size: Desired length of new Buffer. It accepts integer type of data.

Fill: The value to prefill the buffer. The default value is 0.It accepts any of the following: integer, string, buffer type of data.

Encoding: It is Optional.If buffer values are string , default encoding type is utf8.

. Supported values are given below:

(“ascii”,”utf8″,”utf16le”,”ucs2″,”base64″,”latin1″,”binary”,”hex”)

Example :

var zerobuf = Buffer.alloc(20);
console.log(zerobuf);

Here in this example below, it Creates a zero-filled buffer of the specified length, with initializing all the value to fill or 0.

Here we have allocated the memory of 20 bytes or size of 20 bytes to buffer object.Here we have not specified the fill value thus by default it will take zero value in hexadecimal format.

Run command :

C:\Users\AKASH\Desktop\Buffer>node buffer.js

Result :

<Buffer 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00>

Note: The value of unsafe buff is not emptied may contain data from an older buffer, thus to empty the buffer using this example below.

Example 2a:

var buf2 = Buffer.alloc(20, 0b100);
console.log(buf2);

Here we have provided a binary fill of value 4 (0b100)

Run command:

C:\Users\AKASH\Desktop\Buffer>node buffer.js

Result :

<Buffer 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04>

Example 2b:

var buf1 = Buffer.alloc(20, "hello");
console.log(buf1);

Here we have provided a string fill (“hello”)

Run command :

C:\Users\AKASH\Desktop\Buffer>node buffer.js

Result:

<Buffer 68 65 6c 6c 6f 68 65 6c 6c 6f 68 65 6c 6c 6f 68 65 6c 6c 6f>
Buffer.allocUnsafe()

This method creates new buffer object of the specified size but it will not initialize the values thus it is the no-prefill buffer. The segment of allocated memory is uninitialized, an allocated segment of memory might contain sensitive or confidential data from older buffers.Buffer created by Buffer.allocUnsafe() without completely overwriting the memory can make this old data to be leaked when the Buffer memory is read.If you know you’re going to fill up the buffer immediately then you can use allocUnsafe it is more efficient but does not clear out random unused bytes from the buffer.

To secure this buffer from containing old data, you can use the Buffer.fill() method to pre-fill the buffer.

Syntax: The syntax of Buffer.alloc() method is given below

 Buffer.allocUnsafe(size);  

Size : Desired length of new Buffer. It accepts integer type of data.

Example 1: 

var unsafebuff = Buffer.allocUnsafe(30);
console.log(unsafebuff);

Run command :

C:\Users\AKASH\Desktop\Buffer>node buffer.js

Result :

<Buffer 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 50 d9 3f e8 b6 00 00 00 50 d9 3f e8 b6 00>

Note: The value of unsafe buff is not emptied may contain data from older buffer, thus to empty the buffer use this Example below

Example 2:

unsafebuff.fill(0);
console.log(unsafebuff);

Result :

<Buffer 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00>

Buffer Class

Buffer from a given array:

Syntax: Below here is Syntax to create a buffer from a given array.

 Buffer.from([ ]) 

Example1:

var bufarray1 = Buffer.from([100, 599, 300, 122, 500]);
console.log("Total array elements = "+  bufarray1.length);

Run command :

C:\Users\Magnet Brains\Desktop\Buffer>node bufferclass.js

Result :

Total array elements = 5

Example2:

var bufarray2 = Buffer.from([0x68, 0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x77, 0x6f, 0x72,0x6c, 0x64]);
console.log(bufarray2);

Run command :

C:\Users\Magnet Brains\Desktop\Buffer>node bufferclass.js

Result :

<Buffer 68 65 6c 6c 6f 20 77 6f 72 6c 64>
Buffer from a given string:
Buffer.from()

This method is used to generate a buffer from a string, object, array or buffer.

Syntax: The syntax of Buffer.from() method is given below

 Buffer.from(obj, encoding)  

Obj : Object by which should be filled in, Different types that we can fill are (String,Array,Buffer,arrayBuffer)

Encoding : The encoding of string. Default value is utf8. This is an optional parameter.

Example : 

var buf3 = Buffer.from("Magnet-Brains");
console.log("buffer-from : " + buf3);

Run command :

C:\Users\Magnet Brains\Desktop\Buffer>node bufferclass.js

Result : 

buffer-from : Magnet-Brains

Buffer Operations:

Writing to Buffer:

Syntax :  buf.write(string, offset, length, encoding)  

String: String data that is to be written inside buffer

Offset: Number of bytes to skip before starting to write the string. Default: 0  or It is the index of the buffer from where we will start writing it. The default offset value is 0.

Length: That is Number of bytes to write. Default: buf.length – offset

Encoding: That means encoding type to be used. The ‘utf8’ is the default encoding.

Note: Write Method returns the number of octets that are written. If there no sufficient space in the buffer to fit full string, then it will write a part of the string as the return value

Example1:

var buf2 = Buffer.from('hello');
buf2.write('ii',2);
console.log(buf2.toString());

Run command :

C:\Users\Magnet Brains\Desktop\Buffer>node buffoperations.js

Result :

heiio

Example2:

var buf1 = Buffer.alloc(256);
var length = buf1.write("Concept of Buffers in Node.js");
console.log("Octets written : "+  length);

Run command : 

C:\Users\Magnet Brains\Desktop\Buffer>node buffoperations.js

Result :

Octets written : 29

Reading from Buffer:

Given Below is the syntax of the class method which is used to read from a Node Buffer.

Syntax : buf.toString(encoding, start, end) 

Encoding : This is encoding type to be used. The ‘utf8’ is the default encoding type.

Start: This denotes the beginning index to start reading. The defaults value is 0.

End: This denotes the end index to end reading. The default value is the complete buffer.

Example 1:

var buf3 = Buffer.alloc(26);
for (var count = 0 ; count < 26 ; count++) {
    buf3[count] = count + 97;
}
console.log( buf3.toString('ascii'));      
console.log( buf3.toString('ascii',23,52));  
console.log( buf3.toString('utf8',0,11));  
console.log( buf3.toString('utf16le',0,11));
console.log( buf3.toString('ucs2',0,11));
console.log( buf3.toString('base64',0,11));
console.log( buf3.toString('hex',0,11));
console.log( buf3.toString(undefined,0,5)); // encoding will defaults to 'utf8'

Run command :

C:\Users\Magnet Brains\Desktop\Buffer>node buffoperations.js

Result :

abcdefghijklmnopqrstuvwxyz
 xyz
 abcdefghijk
 扡摣晥桧橩
 扡摣晥桧橩
 YWJjZGVmZ2hpams=
 6162636465666768696a6b
 abcde

Example 2 :

var buf4 = Buffer.from('abc');
console.log(buf4.toString());
console.log(buf4);

Run command : 

C:\Users\Magnet Brains\Desktop\Buffer>node buffoperations.js

Result :

abc
<Buffer 61 62 63>
Convert Buffer to JSON:

Given below is the syntax of the class method which is used to convert a Node Buffer into JSON object. The value returned is a JSON-representation of the Buffer instance.

Syntax:  buf.toJSON() 

Example :

var buf5 = Buffer.from('Hello magnet!');
var json = buf5.toJSON(buf5);
console.log(json);

Run command :

C:\Users\Magnet Brains\Desktop\Buffer>node buffoperations.js

Result : 

{ type: 'Buffer',
  data: [ 72, 101, 108, 108, 111, 32, 87, 111, 114, 108, 100, 33 ] }
Concatenate Buffers:

Below Given is the Syntax of the class method which is used to concatenate Node buffers to a single Node Buffer.

Syntax :  Buffer.concat(list, Length) 

List : This is  the array List of the Buffer objects which are to be concatenated.

Length: Here it denotes the total length of the buffers when they are concatenated, it is optional.

Example :

var buffer1 =  Buffer.from('Hi');
var buffer2 =  Buffer.from('Magnet-Brains');
var buffer3 = Buffer.concat([buffer1,buffer2]);
console.log("buffer3 content are: " + buffer3.toString());

Run command :

C:\Users\Magnet Brains\Desktop\Buffer>node buffoperations.js

Result : 

buffer3 content are: HiMagnet-Brains
Compare Buffers:

The compare() method compares two buffer objects and it returns a number according to their differences

Below Given is the Syntax of the class method which is used to  compare two node Buffers

Syntax : 

 buf.compare(buffer1,buffer2) ; buffer1.compare(buffer2)  

Note: This method compares two buffer objects and returns a number defining their differences:

        Shows 0 if they are equal

        Shows 1 if buffer1 is higher than buffer2

        Shows -1 if buffer1 is lower than buffer2

Example 1:

var bufcom1 = Buffer.from('Canada');
var bufcom2 = Buffer.from('United States');
var result = buffer1.compare(bufcom2);
 
console.log("Value returned is : " + result);
if(result < 0) {
  console.log(bufcom1 +" lower than " + bufcom2);
}
else if(result === 0){
  console.log(bufcom1 +" is same as " + bufcom2);
}
else {
  console.log(bufcom1 +" higher than " + bufcom2);
}

Run command :

C:\Users\Magnet Brains\Desktop\Buffer>node buffoperations.js

Result :

Value returned is : -1
Canada lower than United States

Example 2:

var buf1 = Buffer.from('Magnet');
var buf2 = Buffer.from('Magnet');
var x = Buffer.compare(buf1, buf2);
console.log(x);

var buf1 = Buffer.from('ab');
var buf2 = Buffer.from('ba');
var x = Buffer.compare(buf1, buf2);
console.log(x);

var buf1 = Buffer.from('ba');
var buf2 = Buffer.from('ab');
var x = Buffer.compare(buf1, buf2);
console.log(x);

Run command :

C:\Users\Magnet Brains\Desktop\Buffer>node buffoperations.js

Result :

0
-1
1
Copy Buffer:

Below here is the syntax that is used to copy a node buffer.

Syntax : buf.copy(targetBuffer, targetStart, sourceStart, sourceEnd)  

TargetBuffer : This shows where buffer will be copied.

TargetStart: This shows the number that specifies the target buffer start offset. It is an Optional field. The default value is 0.

TargetBuffer: This denotes buffer object where buffer will be copied.

TargetStart: This shows the number that specifies the target buffer start offset. It is an Optional field. The default value is 0.

Example :

var sourceBuff = Buffer.from('Hello World!');
var targetBuff = Buffer.alloc(12);
sourceBuff.copy(targetBuff);
console.log("targetBuff content are : " + targetBuff.toString());

Run command :

C:\Users\Magnet Brains\Desktop\Buffer>node buffoperations.js

Result :

targetBuff content are : Hello World!

 

Slice Buffer:

Below Given is the syntax of the class method which is used to get a sub-buffer of a node buffer.

Syntax : buffer.slice(start, end);  

Start : Here Comes a number that specifies the start of the buffer. It is an optional field with default value as 0.

End: Here Comes a number that specifies the end of the buffer. It is an optional field with default value as buffer.length

Note: Method returns a new buffer the same reference memory of this is same as the old one, but the offset is cropped between the start (defaults to zero) and end (defaults to buffer.length) indexes.The negative indexes start from the end of the buffer.

Example :

var buffer1 = Buffer.from('Czechoslovakia');
var buffer2 = buffer1.slice(6,15);
console.log("buffer2 content are : " + buffer2.toString());

Run command :

C:\Users\Magnet Brains\Desktop\Buffer>node buffoperations.js

Result :

buffer2 content are : slovakia

 

Buffer Length:

Below given is the syntax of the class method which is used to get a size of a node buffer in bytes.

Syntax :   buf.length; 

It returns of buffer in Bytes

Example :

var buffer = Buffer.from('Magnet-brains');
console.log("The length of buffer is : " + buffer.length);

Run command : 

C:\Users\Magnet Brains\Desktop\Buffer>node buffoperations.js

Result :

The length of buffer is : 13

 

Buffer fill:

Below given is the syntax of the class method which is used to fill the buffer with a specified value.

Syntax : buf.fill(value); 

Example :

var buf = Buffer.alloc(7).fill('Hi Guys');
console.log(buf.toString());

Run command :

C:\Users\Magnet Brains\Desktop\Buffer>node buffoperations.js

Result :

Hi Guys

Buffer Index:

Below given is the syntax of the class method which is used to check whether the buffer contains a specified value. If the value is present it will return the index of the first occurrence of the value, otherwise, it will return – 1.

Syntax :  buf.indexOf(value);  

Example :

var buffindex = Buffer.from('Pabbly');
console.log(buffindex.indexOf('b'));

Run command :

C:\Users\Magnet Brains\Desktop\Buffer>node buffoperations.js

Result :

2

 

Buffer Equals:

Below given is the syntax of the class method which is used to compare 2 buffers. It returns true if buffers match, otherwise it will return false .

Syntax : buf.equals(otherbuffer);  

Example :

var buff1 = Buffer.from('pabbly');
var buff2 = Buffer.from('pabbly');
console.log(buff1.equals(buff2));

Run command : 

C:\Users\Magnet Brains\Desktop\Buffer>node buffoperations.js

Result :

true;

Learn More-