- Latest
mongochangestream
- Fix for downstreamprocessRecords
mutation.
- Latest
mongochangestream
- AddlastFlush
tostats
event.
- Breaking Change: When there are multiple
overrides
that match the same path (e.g.*
andfoo.*
both match the pathfoo.bar
), they will now be applied in sequence, where the output of each override is passed as input to the next. - Added a
mapSchema
option as an "escape hatch" to allow preprocessing each node in the input object (usingmap
fromobj-walker
) before converting it to a Crate schema.
- Bump
mongochangestream
to enable "pausable" feature. - Bumped deps.
- Bump packages, including latest
prom-utils
which allows for throttling of items/sec and bytes/sec.
- Updated
createIndexIgnoreMalformed
- droppedbody
field.
- Latest
mongochangestream
- Fixed omit bugs (not relevant to mongo2elastic). - Fixed:
processChangeStream
options should be optional. - Changed testing library from jest to vitest.
- Bumped peer dependencies for
elasticsearch
,ioredis
, andmongodb
.
- Latest
mongochangestream
- Support for parallel syncing of a collection.
- Latest
mongochangestream
- Bug fixes.
- Remove
updateDescription
since it is not needed and may cause aBSONObjectTooLarge
error.
- Fix
passthrough
when set for an array field. - Better
passthrough
logic fortext
fields.
- Removed
options.mapper
in favor ofoptions.rename
which takes an object of dotted path to renamed dotted path. Therename
option is available for syncing and converting a schema to a table._id
mapped to_mongoId
by default.
- Added
operationCounts
to theprocess
event.
- Latest
mongochangestream
- Change stream optionoperationTypes
(insert
,update
,delete
, ...).
processChangeStream
now batches records. Default timeout before the queue is automatically flushed is 30 seconds.
- Latest
mongochangestream
- More robust error code handling formissingOplogEntry
.
- Latest
mongochangestream
- Don't emit thecursorError
event when stopping.
- Latest
mongochangestream
- FSM bug fix.
- Latest
mongochangestream
- Drop health check code in favor ofcursorError
event.
- Latest
mongochangestream
- extend event types.
- Latest
mongochangestream
-runInitialScan
pipeline.
- Emit errors that happened during initial bulk sync
- Optionally pass
mapper
for an override.
- Glob expressions supported for overrides
path
. - Pass any key/value pair in
Override
object.
- Latest
mongochangestream
- Handle master failover scenario properly for initial scan.
- Latest
mongochangestream
- LongermaxSyncDelay
default.
- Renamed
ignoreMalformed
tocreateIndexIgnoreMalformed
. - More info when emitting.
- Export
detectResync
.
- Latest
mongochangestream
- More robust cursor consumption.
- Latest
mongochangestream
- Bug fix.
- Latest
mongochangestream
- generic emitter. - Use emitter from
mongochangestream
which now emits two events on its own.
- Latest
mongochangestream
- health checks.
- Added
passthrough
option to allow for customization of field mappings. - Use
_.merge
instead of flattened key path.
- Use flattened key path for
ignore_malformed
.
- Add optional settings that can be added to
ignoreMalformed
.
- Unwrap first value of multi-valued BSON type. E.g.,
['string', 'null']
=>'string'
.
- Added options for
omit
andoverrides
toconvertSchema
. - Convert string enum to keyword.
- Added
createMappingFromSchema
. convertSchema
bug fixes.
- Added
convertSchema
fn that converts a MongoDB JSON schema to an Elasticsearch mapping.
- Forgot to bump
prom-utils
in this repo.
- Latest
mongochangestream
-batchBytes
option.
- Return
emitter
with events:process
anderror
. - Stats are no longer logged.
- Removed
clearCompletedOn
. - Latest
mongochangestream
-JSONSchema
type.
- Latest
mongochangestream
- Option to strip metadata from a JSON schema fordetectSchemaChange
.
- Latest
mongochangestream
- Ensure that you can callstart
after callingstop
.
- Fix
Document
type. - Bump
mongodb
peer dep. - Latest
mongochangestream
.
- Bug fix from
mongochangestream
.
- Latest
mongochangestream
.
- Latest
mongochangestream
.
- Pass along
reset
and othermongochangestream
fns.
- Latest
mongochangestream
. detectSchemaChange
.
- Lower required elasticsearch dep.
- Added peer dependencies.
- Latest
mongochangestream
.
- Latest
mongochangestream
.
- Initial release.