Skip to content

Commit

Permalink
Add PrintBuffer classes (#2873)
Browse files Browse the repository at this point in the history
Using ArduinoJson to serialize directly to a file stream performs poorly because writes are done byte-by-byte. This issue is mentioned under [performance](https://arduinojson.org/v6/api/json/serializejson/).

The mentioned [ArduinoStreamUtils](https://github.com/bblanchon/ArduinoStreamUtils) library is a bit hefty and doesn't particular fit with the existing Sming stream classes.

This PR adds a very simple `PrintBuffer` class which operates on a `Print` output and can be employed very easily if additional generic buffering is required.

Note that there is little benefit to using this with other memory-based streams so best left to the library or application to decide when its appropriate.
  • Loading branch information
mikee47 authored Aug 7, 2024
1 parent 16cf83d commit f8d10e6
Show file tree
Hide file tree
Showing 3 changed files with 213 additions and 0 deletions.
43 changes: 43 additions & 0 deletions Sming/Core/Data/Buffer/PrintBuffer.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
/****
* Sming Framework Project - Open Source framework for high efficiency native ESP8266 development.
* Created 2015 by Skurydin Alexey
* http://github.com/SmingHub/Sming
* All files of the Sming Core are provided under the LGPL v3 license.
*
* PrintBuffer.cpp
*
****/

#include "PrintBuffer.h"

size_t BasePrintBuffer::write(uint8_t c)
{
buffer[writeOffset++] = c;
if(writeOffset == bufferSize) {
flush();
}
return 1;
}

size_t BasePrintBuffer::write(const uint8_t* data, size_t size)
{
size_t written{0};
while(size != 0) {
auto copySize = std::min(bufferSize - writeOffset, size);
memcpy(&buffer[writeOffset], data, copySize);
writeOffset += copySize;
written += copySize;
data += copySize;
size -= copySize;
if(writeOffset == bufferSize) {
flush();
}
}
return written;
}

void BasePrintBuffer::flush()
{
output.write(buffer, writeOffset);
writeOffset = 0;
}
155 changes: 155 additions & 0 deletions Sming/Core/Data/Buffer/PrintBuffer.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
/****
* Sming Framework Project - Open Source framework for high efficiency native ESP8266 development.
* Created 2015 by Skurydin Alexey
* http://github.com/SmingHub/Sming
* All files of the Sming Core are provided under the LGPL v3 license.
*
* PrintBuffer.h
*
****/

#pragma once

#include <Data/Stream/ReadWriteStream.h>
#include <memory>

/**
* @brief Generic write-through buffer class
* @ingroup stream
* @note Call flush() at end of write operation to ensure all data is output
* This is done automatically when the buffer is destroyed.
*/
class BasePrintBuffer : public Print
{
public:
/**
* @brief Create buffer
* @param output Destination stream
* @param buffer buffer to use
* @param size Size of buffer
*/
BasePrintBuffer(Print& output, uint8_t buffer[], size_t bufferSize)
: output(output), buffer(buffer), bufferSize(bufferSize)
{
}

~BasePrintBuffer()
{
flush();
}

size_t write(uint8_t c) override;

size_t write(const uint8_t* data, size_t size) override;

/**
* @brief Write any buffered content to output
*/
void flush();

private:
Print& output;
uint8_t* buffer;
size_t bufferSize;
size_t writeOffset{};
};

/**
* @brief Write-through buffer using stack only
* @tparam size Size of buffer
*
* Example usage:
*
* FileStream stream("file.txt", File::ReadWrite);
* {
* StaticPrintBuffer<256> buffer(stream);
* writeSomeData(buffer);
* } // Buffer flushed and destroyed when it goes out of scope
*/
template <size_t size> class StaticPrintBuffer : public BasePrintBuffer
{
public:
/**
* @brief Construct a stack-based buffer
* @param output Print destination
*/
StaticPrintBuffer(Print& output) : BasePrintBuffer(output, buffer, size)
{
}

private:
uint8_t buffer[size];
};

/**
* @brief Write-through buffer using heap storage
*
* Example usage:
*
* FileStream stream("file.txt", File::ReadWrite);
* {
* HeapPrintBuffer buffer(stream, 512);
* writeSomeData(buffer);
* } // Buffer flushed and destroyed when it goes out of scope
*/
class HeapPrintBuffer : public BasePrintBuffer
{
public:
/**
* @brief Construct a stack-based buffer
* @param output Print destination
* @param size Buffer size
*/
HeapPrintBuffer(Print& output, size_t size) : HeapPrintBuffer(output, new uint8_t[size], size)
{
}

private:
HeapPrintBuffer(Print& output, uint8_t* buffer, size_t size) : BasePrintBuffer(output, buffer, size), buffer(buffer)
{
}

std::unique_ptr<uint8_t[]> buffer;
};

/**
* @brief Write-through buffer using heap storage and owned stream pointer
*
* Example usage:
*
* auto stream = std::make_unique<FileStream>("file.txt", File::ReadWrite);
* auto bufferedStream = new DynamicPrintBuffer(std::move(stream), 512);
*
* // write to bufferedStream as required via callbacks, etc.
* ...
*
* // This destroys both buffer *and* the file stream
* delete bufferedStream;
*/
class DynamicPrintBuffer : public BasePrintBuffer
{
public:
/**
* @brief Construct a stack-based buffer
* @param output Print destination, will take ownership of this
* @param size Buffer size
*/
DynamicPrintBuffer(std::unique_ptr<Print>&& output, size_t size)
: DynamicPrintBuffer(output.release(), new uint8_t[size], size)
{
}

~DynamicPrintBuffer()
{
flush();
}

private:
DynamicPrintBuffer(Print* output, uint8_t* buffer, size_t size)
: BasePrintBuffer(*output, buffer, size), output(output), buffer(buffer)
{
}

std::unique_ptr<Print> output;
std::unique_ptr<uint8_t[]> buffer;
};
15 changes: 15 additions & 0 deletions docs/source/framework/core/data/buffering.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
Buffering
=========

In general, writing efficiently to files is best done in chunks, such as by building a line of data in a :cpp:class:`String` and writing it in one go.

Sming offers a simple write-through buffering mechanism which can be used where necessary. The :library:`ArduinoJson` can benefit greatly from this.

.. doxygenclass:: StaticPrintBuffer
:members:

.. doxygenclass:: HeapPrintBuffer
:members:

.. doxygenclass:: DynamicPrintBuffer
:members:

0 comments on commit f8d10e6

Please sign in to comment.