Make import of 'user' data possible

[jomeier]

Hi,

I'm working on an tower export/import script and try to use tower-cli for that.

Here: #581

I describe a problem, where the export of 'user' data and the reimport with the same! data doesn't work. Some required fields (like email, first and last name, ...) are missing in the export.

This PR should fix that. For some reason some required fields are not exported with:

tower-cli receive --user all > user.json

Best regards,

Josef

[coveralls]

Coverage remained the same at 63.827% when pulling e4b12c6 on jomeier:additional-exported-user-fields into 9a22336 on ansible:master.

[decentral1se]

Sollid stuff @jomeier!

[jomeier]

@john-westcott-iv
Any news about the code review? Some people seem to have the same problems. It would be great to get feedback. Thx.

[john-westcott-iv]

My apologies for the delay, I have been exceptionally busy over the last month. I'm not sure this is going to be the best way to fix this issue, and I think we still need to do some more troubleshooting to determine is something is going awry with your AWX installation as most users don't seem to be getting this problem.

The way the receive process is supposed to work is that TowerCLI should be asking the API about the fields for a given asset type (i.e. a user) and then determining what to output based on this. To get the required field information TowerCLI sends an HTTP OPTIONS call to the assets API end point. In the case of a user object it would be an OPTIONS call to /api/v2/users. You can do this in the UI by doing a GET on /api/v2/users and then clicking the OPTIONS button in the upper right. The result should look something like this:

{
    "name": "User List",
    "description": "# List Users....",
    "renders": [
        "application/json",
        "text/html"
    ],
    "parses": [
        "application/json"
    ],
    "actions": {
        "POST": {
            "username": {
                "type": "string",
                "required": true,
                "label": "Username",
                "help_text": "Required. 150 characters or fewer. Letters, digits and @/./+/-/_ only.",
                "max_length": 150
            },
            "first_name": {
                "type": "string",
                "required": false,
                "label": "First name",
                "max_length": 30
            },
           ...

TowerCLI then uses the POST action to decide which fields are required to build a new user (based on the required tag) and what the default value is for any optional fields.

After it gets this data, TowerCLI makes an API call to get the users themselves (via stepping through the pages of GET requests to /api/v2/users/). For each user it receives it will look at the fields of the user and match those fields to the fields in the OPTIONS call. A required field is always added to the output; an optional field whose value does not match the default value is also printed in the output.

The API_POST_OPTIONS field you are modifying was intended to handle a very specific case where performing an HTTP OPTIONS call on /api/v2/schedules does not return a POST action. By hard coding a user object in here if the definition of a user asset changed in a future version of AWX TowerCLI would have to be updated in order to handle the new fields.

So again, I think we really need to determine what exactly is going on. In the small amount of time I was able to devote to this issue I did download a couple different versions of AWX but was unable to replicate the issue. Can you post the result of an HTTP OPTIONS call for /api/v2/users? This can be done with curl using this syntax:
curl -v -X OPTIONS http://<server>:<port>/api/v2/users/ --user <username>:<password>

In addition, can you get the data about the admin user from the API with this curl command:
curl -v -X GET http://<server>:<port>/api/v2/users/?username=admin --user <username>:<password>
As stated before, technically TowerCLI will not make this call. Instead if will call /api/v2/users/ and then step through all of the pages concatenating the users into a single response to be processed by the receive command. This task would be difficult to do via curl so the above command should suffice for now.

Please be sure to run these two commands form the same machine you are running TowerCLI from. That way, if there is something between your TowerCLI and AWX installation we will capture the correct results.

As a side note, if run receive with -v it will show you messages regarding what calls it is making to AWX:

tower-cli receive --tower-host http://localhost:8081 --user all -v
OPTIONS http://localhost:8081/api/v2/credentials/

OPTIONS http://localhost:8081/api/v2/job_templates/

OPTIONS http://localhost:8081/api/v2/workflow_job_templates/

OPTIONS http://localhost:8081/api/v2/inventory_scripts/

OPTIONS http://localhost:8081/api/v2/projects/

OPTIONS http://localhost:8081/api/v2/credential_types/

OPTIONS http://localhost:8081/api/v2/users/

*** DETAILS: Getting records. *************************************************
GET http://localhost:8081/api/v2/users/
Params: []

OPTIONS http://localhost:8081/api/v2/teams/

OPTIONS http://localhost:8081/api/v2/organizations/

OPTIONS http://localhost:8081/api/v2/notification_templates/

OPTIONS http://localhost:8081/api/v2/inventories/

[
  {
    "username": "admin", 
    "first_name": "Admin", 
    "last_name": "User", 
    "asset_type": "user", 
    "email": "[email protected]"
  }, 
...

[TwoTwenty]

I get this with
tower-cli receive --tower-host http://host --user all -v

WARNING: Asset type user has no API POST options no pre-checks can be performed

I get this on a fresh install of awx docker images 1.0.8
&Tower CLI 3.3.0

[srgvg]

Seems like a bug in 1.0.x? that is fixed in 2.0.1?

receiving from AWX 1.0.7.2:

➡  tower-cli receive --user all
[
  {
    "asset_type": "user"
  },
  {
    "asset_type": "user"
  }
]

Whilst the same (tower-cli version) on 2.0.1:

➡  tower-cli receive --user all
[
  {
    "asset_type": "user",
    "username": "someuser",
    "first_name": "Serge",
    "last_name": "Van Ginderachter",
    "email": "[email protected]"
  },
  {
    "asset_type": "user",
    "username": "admin",
    "first_name": "",
    "last_name": "",
    "email": "root@localhost",
    "is_superuser": true
  }
]

[john-westcott-iv]

@TwoTwenty Thanks for the post. Are you running the export command as Admin? There may be issues where if you are running as a non-prived user you don't see the POST options.

[gpduck]

I'm having the same problem with the export, however the very first time I ran the command it worked and now (minutes later) it is not working. I've checked the User List options in my browser and it does contain the POST section within actions. I've also tried running the curl options command and also see the POST section. I modified my common.py to dump out the value in return_json and it is indeed missing the POST section.

What I've narrowed it down to is my ~/.tower_cli.cfg file. If I run tower-cli config username and tower-cli config password, then the export command works. However when I run tower-cli login MYUSER and the config file contains a value for oauth_token, the export command does not work. I ran curl using the oauth_token and it returns successfully but without the POST action.

[nicolaibaralmueller]

I'm having the same problem with the export, however the very first time I ran the command it worked and now (minutes later) it is not working. I've checked the User List options in my browser and it does contain the POST section within actions. I've also tried running the curl options command and also see the POST section. I modified my common.py to dump out the value in return_json and it is indeed missing the POST section.

What I've narrowed it down to is my ~/.tower_cli.cfg file. If I run tower-cli config username and tower-cli config password, then the export command works. However when I run tower-cli login MYUSER and the config file contains a value for oauth_token, the export command does not work. I ran curl using the oauth_token and it returns successfully but without the POST action.

I have the exact same issue. Thanks for posting your findings. That worked. Tower-cli login does not work.