Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running tests not against the live HubSpot demo account #40

Open
rheinwerk-mp opened this issue Jun 25, 2019 · 5 comments
Open

Running tests not against the live HubSpot demo account #40

rheinwerk-mp opened this issue Jun 25, 2019 · 5 comments

Comments

@rheinwerk-mp
Copy link
Contributor

My colleagues and I a currently working on extending the contacts API (as mentioned also in #39).

The tests for this (and other) API rely on HubSpot's live demo account. This causes some problems. Mainly the tests are failing at the moment because the demo account has reached its daily limit. We need to wait until the next day, when the limit is reset, to run all tests, hoping is has not been exceeded again.

Furthermore the tests can fail when the live database does not contain the expected records. Two examples: When the database is flushed, all tests relying on the contact from the BASE_CONTACT would fail. And if there are less than 20 records in the database, testing the limit parameter in the test test_get_all will not fail but will not test anything useful either since there will always be less than 20 results, regardless of the parameter value.

As a solution we would suggest to mock the API calls and their responses in the current unit tests. We would include the necessary changes (at least for the contact API tests) in the following PR.

Let me know what do you think.

@jpetrucciani
Copy link
Owner

Thanks for bringing this issue up!

Yeah it'd probably be best to mock those all out - I had started on rebuilding the test suite and quickly ran into all sorts of problems trying to get how I wanted to test things working (including a nasty workaround that you've found in a couple different places for loading a record on boot to use for testing

# since we need to have an id to submit and to attempt to get a contact,
# we need to be hacky here and fetch one upon loading this file.
BASE_CONTACT = CONTACTS.get_all(limit=1)[0]
).

Good point on the test_get_all tests as well - there's a lot of the testing that I hadn't completely thought through when I started writing the tests!

I'd agree with you that mocking the calls would probably be best - I'm open to any methods of doing so that you guys are comfortable with!

@jpetrucciani
Copy link
Owner

I've also created a topic on the HubSpot developer forum about this, and asked if we could potentially have a different key specific for testing - I do feel that mocking it is fine, but not entirely optimal. If we could be hitting the real API during tests, that'd allow us to test for any changes that they've made as well.

@rheinwerk-mp
Copy link
Contributor Author

Great that you are open changing the tests.

Regarding testing against the live API: Yes, but I would suggest to create a different set of tests (integration tests) for this purpose.
The unit tests on the other hand would only test the integrity of the code itself.

@jpetrucciani
Copy link
Owner

Good point - that makes a lot of sense!

@jpetrucciani
Copy link
Owner

As of the latest commit/release 3.2.17, I've included my test account key in the tests, so we should be able to use that key for integration tests (so at least we can get the old tests back up and running on Travis!). I've also switched the CI to use tox across the three supported versions we've specified.

I'm still on board with also building out a bunch of unit tests for the library that use your mocking methods - I'll just start slowly writing out some new tests when I get some more time!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants