pfioh command: pullPath using zip

pfioh command: pullPath (zip)

Abstract

This page describes the pullPath command to pfioh. It is used to pull a directory tree to a remote server. In this example, the tree is zipped up at the remote server, transferred locally, and then unzipped at the specified local destination.

Preconditions

• A HOST_IP environment variable that denotes the IP of the host housing the service. In Linux, you can do:

export HOST_IP=$(ip route | grep -v docker | awk '{if(NF==11) print $9}')

• Make sure that pfioh has been started (see here for more info):

pfioh --forever --httpResponse --storeBase=/tmp --createDirsAsNeeded

msg summary

The msg payload of the REST interaction with pfioh is:

{  "action": "pullPath",
    "meta": {
        "remote": {
            "path":         "/usr/sbin"
        },
        "local": {
            "path":         "/tmp",
            "createDir":    true
        },
        "transport": {
            "mechanism":    "compress",
            "compress": {
                "encoding": "base64",
                "archive":  "zip",
                "unpack":   true,
                "cleanup":  true
            }
        }
    }
}

pullPath

pfurl calling syntax

Assuming satisfied preconditions, let's pull a directory tree from the remote machine/host to the remote local machine:

pfurl --verb POST --raw --http ${HOST_IP}:5055/api/v1/cmd --msg \
'{  "action": "pullPath",
    "meta": {
        "remote": {
            "path":         "/usr/sbin"
        },
        "local": {
            "path":         "/tmp/sbin",
            "createDir":    true
        },
        "transport": {
            "mechanism":    "compress",
            "compress": {
                "encoding": "base64",
                "archive":  "zip",
                "unpack":   true,
                "cleanup":  true
            }
        }
    }
}' --quiet --jsonpprintindent 4  

Using the dockerized version of purl

To use the dockerized version of pfurl, and assuming a pfioh on the given IP:

Using the source repo

If you have cloned the source repo, you can cd to the root directory and execute the docker helper scripts in the docker-bin directory.

docker-bin/pfurl --verb POST --raw --http 172.17.0.2:5055/api/v1/cmd --msg \
'{  "action": "pullPath",
    "meta": {
        "remote": {
            "path":         "/usr/sbin"
        },
        "local": {
            "path":         "/tmp"
        },
        "transport": {
            "mechanism":    "compress",
            "compress": {
                "encoding": "base64",
                "archive":  "zip",
                "unpack":   true,
                "cleanup":  true
            }
        }
    }
}' --quiet --jsonpprintindent 4  

Calling docker directly

The helper script just creates a docker run command line string. You can run this string directly without using the helper script and directly calling the docker app:

docker run --name pfurl -v /home:/Users --rm -ti fnndsc/pfurl --verb POST --raw --http 172.17.0.2:5055/api/v1/cmd --msg \
'{  "action": "pullPath",
    "meta": {
        "remote": {
            "path":         "/usr/sbin"
        },
        "local": {
            "path":         "/tmp"
        },
        "transport": {
            "mechanism":    "compress",
            "compress": {
                "encoding": "base64",
                "archive":  "zip",
                "unpack":   true,
                "cleanup":  true
            }
        }
    }
}' --quiet --jsonpprintindent 4  

This will pull the remote /usr/sbin directory to the local machine's /tmp directory. Note that the zip compression will not preserve file mode bits (executable files need to have their execute bit explicitly set again).

return payload

The above call returns the JSON string:

{
    "stdout": {
        "remoteCheck": {
            "size": "48",
            "msg": "Check on remote path successful.",
            "response": {
                "isdir": true,
                "status": true,
                "isfile": false
            },
            "timestamp": "2017-03-13 14:42:09.973829",
            "status": true
        },
        "localCheck": {
            "msg": "Check on local path successful.",
            "check": {
                "isdir": true,
                "status": true,
                "isfile": false
            },
            "status": true,
            "timestamp": "2017-03-13 14:42:09.969986"
        },
        "status": true,
        "msg": "unzip operation successful",
        "compress": {
            "remoteServer": {
                "size": "1,372,100",
                "msg": "PULL successful",
                "response": "<truncated>",
                "timestamp": "2017-03-13 14:42:10.235504",
                "status": true
            },
            "localOp": {
                "unzip": {
                    "path": "/tmp",
                    "msg": "unzip operation successful",
                    "fileProcessed": "/tmp/sbin.zip",
                    "status": true,
                    "timestamp": "2017-03-13 14:42:10.302147",
                    "filesize": "1,028,681",
                    "zipmode": "r"
                },
                "decode": {
                    "msg": "Decode successful",
                    "fileProcessed": "/tmp/sbin.zip",
                    "status": true
                }
            },
            "status": true,
            "msg": "unzip operation successful"
        }
    }
}

oneShot

If --oneShot is passed to purl, then a server control message is transmitted after the file IO event that effectively shuts down the remote server:

pfurl --verb POST --http 172.17.0.2:5055/api/v1/cmd/ --oneShot --msg \

--30--