Releasing updates to NPM?

[stieg]

Hey Devs,

So was setting this up and hit issues. Eventually figured out that what is on NPM is extremely old (v0.0.1) vs what you have in the project here (v0.1.2). So I'm curious if you all plan to release this to NPM or not?

Also... off topic but is there any reason why this hasn't just been merged into the mongoose project. Seems important enough to warrant being in there but then again what do I know? TIA.

[vkarpov15]

You're looking at the mongoose-double npm package, that one is out of date and should be deprecated. We publish updates to this repo as @mongoosejs/double: https://www.npmjs.com/package/@mongoosejs/double.

And in my experience this project isn't important enough to merge into mongoose core - it is very rare that as a JS developer it matters whether the db stores properties as integers or doubles, because they become numbers in JavaScript either way. What use cases are you using this project for?

[stieg]

You're looking at the mongoose-double npm package, that one is out of date and should be deprecated. We publish updates to this repo as @mongoosejs/double: https://www.npmjs.com/package/@mongoosejs/double.

I did eventually figure this out though it was a bit confusing since all the other packages follow this naming format and this one was here. Would be nice of the original maintainer to turn over control of that npm package name to you (since it is the same package anyways; just outdated).

And in my experience this project isn't important enough to merge into mongoose core - it is very rare that as a JS developer it matters whether the db stores properties as integers or doubles, because they become numbers in JavaScript either way.

While I agree with your assessment that this is a less common path for developers I would disagree that it does not belong in mongoose. Rationale being that because mongoose is the package that enables a developer to dictate the type to be stored in the DB, having just a generic number type on its own is insufficient because the user is unable to strongly dictate what type is actually being stored. A saner solution would be to prioritize strong types that map between javascript and the native BSON types. Then it makes sense at a secondary level to expose types like number that behave the way that they do because most developers don't really care. But that is the opinion of one developer; YMMV.

What use cases are you using this project for?

We use mongoose as the back-end to our API services primarily. Controlling the types in Mongo has become a priority for us because we map our MongoDB into a PostgreSQL DB for further data analysis. When the type in Mongo is variable then the mapping software starts renaming columns in the PostgreSQL DB to deal with the ambiguity, which in turn makes mapping and analysis much more difficult. As a workaround we have written a set of scripts that correct the data types in Mongo before the mapping software runs but this is an imperfect and temporary solution until we can use the mongoose-double package.

[vkarpov15]

Thanks for the heads up, I deprecated mongoose-double on npm.

We may investigate an option to store JavaScript numbers consistently as doubles in MongoDB, your argument makes sense. The decision for the mongodb driver to store numbers as ints where possible long pre-dates my involvement with the project - if I were making that decision I would've made a different one, but it's hard to change a decision that's been baked in for almost a decade.

In email you mentioned you were using stitch. Can you take a look at purr? That's the in house ETL that my current employer built out for piping mongodb -> postgres. I haven't worked with it so I can't vouch for how easy it is to set up, but it works well for us. And it handles the int vs double distinction.

[vkarpov15]

Hi, I don't think Purr has support for either of those. Our ETL setup is really more of an EL, we do minimal transformation of the data between MongoDB and Postgres. Feel free to open up an issue on the Purr repo though :) Thanks, Valeri On Fri, Jan 17, 2020 at 5:21 PM Andrew Stiegmann <[email protected]> wrote:

…

Hey Valeri, Well after a whole slew of back and forth with Stitchdata we are back to looking for other ETL solutions to load MongoDB into Postgres. Just gave purr a go on my local machine. Setup wasn't too bad but there are some issues we are seeing our our end. One thing that I can't quite figure out is if Purr is capable of handling fields within objects and mapping them to column. For example if I have an airport collection that looks like this: ``` { name: "foo", location: { latitude: 12.34, longitude: 56.78 } } ``` I don't see an easy way to map the latitude and longitude fields to something like `location__latitude` and `location__longitude` respectively. I have set my source value in the collections file to `location.latitude` in example but it does not seem to understand it. Any ideas? Another thing I can't figure out is if Purr is capable of unrolling arrays into another table or not. Based on what I see it appears to not be capable of this since there is no mention of it in the docs. Would love to hear how you all overcame these shortcomings (if you did or if you worked around them in a different way). We probably have our data structured in such a way that it makes applying an ETL to our MongoDB a royal PiTA. Cheers. On Thu, Nov 14, 2019 at 5:56 AM Valeri Karpov ***@***.***> wrote: > Thanks for the heads up, I deprecated mongoose-double on npm > <https://www.npmjs.com/package/mongoose-double>. > > We may investigate an option to store JavaScript numbers consistently as > doubles in MongoDB, your argument makes sense. The decision for the mongodb > driver to store numbers as ints where possible long pre-dates my > involvement with the project - if I were making that decision I would've > made a different one, but it's hard to change a decision that's been baked > in for almost a decade. > > In email you mentioned you were using stitch. Can you take a look at purr > <https://github.com/boosterfuels/purr>? That's the in house ETL that my > current employer built out for piping mongodb -> postgres. I haven't worked > with it so I can't vouch for how easy it is to set up, but it works well > for us. And it handles the int vs double distinction. > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > < #6?email_source=notifications&email_token=AABCRTYQMXSHZ7UOMAAW7ODQTVYMLA5CNFSM4JDFW7TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEECKDLQ#issuecomment-553951662 >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AABCRT6MX5U4I64LIAFN3D3QTVYMLANCNFSM4JDFW7TA > > . > -- Andrew AV. Stiegmann — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#6?email_source=notifications&email_token=AAMLSKKPSBXEO5CN3XGOBA3Q6IVNRA5CNFSM4JDFW7TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJJER3Q#issuecomment-575817966>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMLSKL6P2ISTVI7OVMWDKTQ6IVNRANCNFSM4JDFW7TA> .