What is highWaterMark in Streams? Explained with Code

If you enjoy this story and it helps you understand, give it a like or share!

---

I’m a bartender at a bar, and my job is to serve drinks to customers as efficiently as possible. The highWaterMark in JavaScript streams is like my decision on how many drinks I should prepare in advance to keep the service smooth without overwhelming myself.

In my bar, I have a tray that represents the buffer—the space where I can store prepared drinks. The highWaterMark is like setting a limit on how many drinks I can keep on the tray before I start serving them to customers. This setting helps me balance between being prepared and not having too many drinks that might go to waste.

If I set a low highWaterMark, it’s akin to only making a couple of drinks at a time. This means I might have to rush to make more drinks when the crowd gets thirsty, which could lead to delays in service. On the other hand, if I set a high highWaterMark, I might end up with too many drinks on the tray, risking that they go flat or warm.

Finding the right balance is crucial. It allows me to serve customers promptly without overloading myself with too many prepared drinks. In the same way, setting the highWaterMark in a stream helps manage the flow of data, ensuring the stream is neither too slow to respond nor overwhelmed with too much data at once.

So, just like my strategy to keep the bar running smoothly, the highWaterMark helps a stream manage its data efficiently, ensuring a steady and manageable flow.

---

In JavaScript, streams are used to handle reading and writing of data efficiently. Specifically, the highWaterMark property sets a threshold for when to stop reading data into the buffer and when to resume, similar to how I decide how many drinks to prepare in advance.

Let’s look at an example with a readable stream in Node.js:

const fs = require('fs');

// Create a readable stream with a specific highWaterMark
const readableStream = fs.createReadStream('example.txt', { highWaterMark: 16 * 1024 }); // 16KB

readableStream.on('data', (chunk) => {
  console.log(`Received ${chunk.length} bytes of data.`);
  // Process the chunk of data here
});

readableStream.on('end', () => {
  console.log('No more data to read.');
});

In this example, the highWaterMark is set to 16KB, meaning the stream will read data in chunks of up to 16KB. This allows for efficient data processing without overwhelming the memory.

Now, let’s consider a writable stream:

const writableStream = fs.createWriteStream('output.txt', { highWaterMark: 32 * 1024 }); // 32KB

for (let i = 0; i < 1e6; i++) {
  const canContinue = writableStream.write('Some data\n');
  if (!canContinue) {
    console.log('Backpressure: waiting for drain event.');
    writableStream.once('drain', () => {
      console.log('Resuming write after drain.');
    });
    break;
  }
}

writableStream.end('Final data');

Here, the highWaterMark is set to 32KB for the writable stream. If the buffer reaches this limit, the stream will apply backpressure, pausing the writing process until the buffer has been drained, ensuring that the system isn’t overwhelmed with too much data at once.

Key Takeaways:

1. Buffer Management: The highWaterMark property is crucial for managing the buffer size in streams, ensuring efficient data flow without overloading the system.
2. Backpressure Handling: Properly setting highWaterMark helps handle backpressure, allowing streams to pause and resume data processing as needed.
3. Performance Optimization: By adjusting the highWaterMark, developers can optimize the performance of their applications based on the specific needs and resources available.