trouble setting up photon - app fails/crashes on submitting WiFi credentials

[kneuron]

We have multiple (3, currently) customers who are unable to give WiFi credentials to their photon using our app (CleverPet - using the spark-setup-ios v 0.5.0), and all are failing in a very similar way. Digging in, we found that all of them were on iOS 9.3.2 on an iPhone 6S+ One very helpful customer downloaded the latest Particle app and experienced similar behavior attempting to set up.

Here's the behavior:

1. Photon (running our application firmware on top of 0.4.9 system firmware, system thread enabled) in listening mode.
2. User attempts setup via our app, using the spark-setup-ios. They click 'ready' then go to the screen that describes setting their phone wifi to the Photon's SSID.
3. They set their phone to the photon's wifi, get the dropdown alert that their phone has connected and click on it, bringing them back to the app.
4. The spinny loading icon spins a couple of times, then they are brought to the list of available SSIDs to provide credentials for.
5. They select their WiFi network, provide the password, and submit.
6. The app immediately quits to an error screen (it does not go into the checklist screen, or maybe does VERY briefly) and the photon behavior does not change. It continues blinking blue. It doesn't change its rate of flashing, it doesn't turn green or cyan. The SSID is still available in settings. This has been confirmed via video call with two customers.

Things tried:

• multiple different wifi networks set up on the same router, using different security setups - most started on WPA2, but one guy tried setting up WEP (before we told him not to), and also tried creating an open network.
• resetting credentials on the photon
• Particle app - this one was particularly interesting - One user downloaded the latest version of the Particle app from the app store and saw similar behavior, except that the app didn't get to an error screen, it actually crashes hard and just shuts down whenever he attempts to submit wifi credentials. The photon remains in listening mode.

Things not tried:

• We have not yet tested this on the offending platform iPhone 6s+ iOS 9.3.2
• None of the users have yet tried it on a different phone platform - we have asked them to try and acquire a different phone to test with.
• We have not asked them to try setting it up from safe mode
• Do not have complete router information, but the one customer who we've done the most troubleshooting with has this.

Additional information:
We have many (hundreds) of users who are able to set up just fine with the same firmware running on the photon and the same app. I do not know the numbers, but assume some of them might be using iPhone 6S+ on 9.3.2

The things ctmorrison describes in this post have some similarities to our situation.

I'm not sure if this is the right place to post this sort of troubleshooting - in the spark-setup-ios issues list, so please let me know if it belongs elsewhere.

[kneuron]

has anyone had a chance to look at this? We have multiple users that can't get their devices online.

[philipmeier]

I would be happy to provide more information about the users that encounter this error. More and more people are encountering this situation. Users seem to repeatedly get the following error from the SDK:

SparkSetupMainControllerResultFailureConfigure

For example, some users have more than 10 errors, with the same result each time. We have at least 4 users that never got online, and that we can provide concrete times on when they attempted and failed. We do know the particle ID for these users. Even though the photon stays in listening mode, and returns an error, we do get the particle ID back.

Would this information help you trouble shoot what the problem is? If we get into specific customer emails or particle IDs, should we switch to a more private channel?

[philipmeier]

For example, here is one customer that continues to have the Configuration failure during setup. (in our org, their photon id starts with: 3400200009473434) On the dashboard the photon shows as "never connected." Initially we thought this might be a phone-hardware specific error. Given the increased occurrence of these errors across many phones, we don't think thats as likely any more.

[crenwick]

😞 -- this is rough.

I've definitely seen failures like this in the field, but nothing that caused an iOS crash or immediate error exits. One of the more common failures I've seen is the app spinning forever, trying to connect the device online. For that, we have customers power cycle our devices (which I'm sure you have tried).

Have you tried typing the SSID in manually (even though it shows up in the list)? I've used that trick to fix a few issues before.

I'm curious, are you using enterprise accounts?

[idokleinman]

I don't think it has something to do with enterprise account or with manual SSID entry..
The error you describe here means that the SDK fails at sending the configureAP command to the photon which is the first step to set the Wi-Fi credentials - this usually means that the socket connection between the phone and the photon was dropped before user has gotten to this screen.

I'd love to get a better understand / conditions on how to recreate this scenario so we can debug it better - this will probably involve a firmware AND mobile app/SDK update that we plan to roll out after we improve on setup reliability - but this is tricky to recreate and debug.

[kneuron]

Hey @crenwick Thanks for the tips -- we'll keep them in mind -- and for the commiseration :) To be honest, we've had hundreds of setups that go smoothly, these edge cases are bound to happen - the key for us is responding quickly and getting the customers online as quickly as we can.

@idokleinman thanks for checking this issue out. As @philipmeier mentioned above, we've got a bunch more data we've been collecting from users about who/when/how often; much of it based around the error codes you added to the SDK as of 0.5.0, hopefully sharing some of this data will help us figure out what's going on. I'll reiterate that we've seen similar behavior happen with the Particle app for some of these users - it crashes on attempting to provide credentials at the same point that our app does.

We agree that it's tricky to recreate and debug, our ability to do this might benefit from some tips from you guys on which variables are likely to matter, and which do not. For example, should we be focusing on the app, the phone device, the firmware, the router, or even possibly the particle device hardware itself? We've tried each of these in most cases (well, we have not yet swapped out their particle devices in most cases - that's quite expensive for us, though an option if it's indicated. The couple of times we tried that for some different symptoms, it turned out to be their router after all...). Based on your comments, it seems like configureAP is implicated and we should be focusing on firmware and phone hardware? Ultimately, short of heading to our users' houses and trying it out on their phones, with their photons, we can't recreate all the cases. We might be able to ask them to run some very few tests that might tell us something, based on their willingness/frustration.

What information do you think would help most? We will provide it if we have it, and try to collect it if we don't already.

[idokleinman]

Hi guys, many thanks for your patience on this matter. A quick update: @m-mcgowan & myself will try to make an effort to find the root cause of these issues this sprint. As a general idea our effort will include creating a verbose debug logging version of the 0.4.9 firmware as well as the iOS app (with verbose logging of the Soft AP process) and try to recreate the issues in the SF office and at Mat's place (he's in Europe) and analyze the data we get from the debug info. We might also try to run photons with 0.5.0+ firmware to see if there's an improvement in reliability once we can dependably reproduce the original issues. Let us know if by now you have collected additional information that might help the investigation.