feat!: update to work with w3up api (#89)

Rewrite of the add-to-web3 github action to work with the new w3up web3.storage api. [`w3cli`](https://github.com/web3-storage/w3cli) does everything we need to upload files to web3.storage, so this action now provides a very thin wrapper around that cli in the form of a composite action. ## Breaking changes Auth is based on [UCAN](https://web3.storage/docs/concepts/ucans-and-web3storage/)s now. Use w3cli to create a signing key for this action to use, and create a ucan that delegates upload abilities to that key. - input: `web3_token` has been removed, as token based auth as been removed. - set `secret_key` to a base64 encoded siging key you create via `w3 key create`. - set `proof` to the a base64 CAR encoded UCAN delegating capabilities to the secret_key's DID. - input: `wrap_with_directory` has been removed. w3cli does not wrap directories in directories by default now. Only single files get wrapped to preserve their filename. - set `wrap: false` to disable the wrapping of single files. - input: `web3_api` has been removed. Targeting other services is not currently supported. Open an issue if you need that. - input: `include_hidden` has been renamed to `hidden` to match the w3cli flag name Fixes #87 Needs storacha/w3cli#158 License: MIT --------- Signed-off-by: Oli Evans <[email protected]> Co-authored-by: Alan Shaw <[email protected]>
storacha · Jan 17, 2024 · 6b67289 · 6b67289
1 parent 194f63d
commit 6b67289
Show file tree

Hide file tree

Showing 16 changed files with 150 additions and 44,287 deletions.
diff --git a/.gitattributes b/.gitattributes
diff --git a/.github/dependabot.yml b/.github/dependabot.yml
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
@@ -7,27 +7,45 @@ on:
       - 'releases/*'
 
 jobs:
-  test:
+  run-action:
     runs-on: ubuntu-latest
     steps:
-    - uses: actions/checkout@v2
-    - uses: actions/setup-node@v2
+    - uses: actions/checkout@v4
+
+    - name: run 1 - upload
+      uses: ./
+      id: run1
       with:
-        node-version: 16
-    - uses: bahmutov/npm-install@v1
-    - run: npm run lint
-    - run: npm test
-      env:
-        INPUT_WEB3_TOKEN: ${{ secrets.WEB3_STORAGE_TOKEN }}
+        path_to_add: 'test/fixtures'
+        proof: ${{ secrets.W3_PROOF }}
+        secret_key: ${{ secrets.W3_PRINCIPAL }}
 
-  run-action:
-    runs-on: ubuntu-latest
-    steps:
-    - uses: actions/checkout@v2
-    - uses: ./
+    - name: test run 1 output
+      run: echo "unexpected output cid ${{ steps.run1.outputs.cid }}" && exit 1
+      if: ${{ steps.run1.outputs.cid != 'bafybeifpw7nrh374rzfcxpaw3bkp6fr7djmujg5wvib6ma7i7n76t3k53q' }} 
+
+    - name: run 2 - upload (wrap=false)
+      uses: ./
+      id: run2
+      with:
+        wrap: false
+        path_to_add: 'test/fixtures'
+        proof: ${{ secrets.W3_PROOF }}
+        secret_key: ${{ secrets.W3_PRINCIPAL }}
+
+    - name: test run 2 output
+      run: echo "unexpected output cid ${{ steps.run2.outputs.cid }}" && exit 1
+      if: ${{ steps.run2.outputs.cid != 'bafkreigh2akiscaildcqabsyg3dfr6chu3fgpregiymsck7e7aqa4s52zy' }} 
+
+    - name: run 3 - upload (hidden=true)
+      uses: ./
+      id: run3
       with:
-        web3_api: 'https://api-staging.web3.storage'
-        web3_token: ${{ secrets.WEB3_STORAGE_TOKEN }}
+        hidden: true
         path_to_add: 'test/fixtures'
-        # test non default option
-        include_hidden: true
+        proof: ${{ secrets.W3_PROOF }}
+        secret_key: ${{ secrets.W3_PRINCIPAL }}
+
+    - name: test run 3 output
+      run: echo "unexpected output cid ${{ steps.run3.outputs.cid }}" && exit 1
+      if: ${{ steps.run3.outputs.cid != 'bafybeiht4hxkewkiqwa3rbw6bnlom2s7wajkbspsgjgn6wojioii4rh77y' }} 
diff --git a/README.md b/README.md
@@ -1,62 +1,94 @@
-<h1 align="center">⁂<br/>web3.storage</h1>
-<p align="center">Add a directory to web3.storage from an Action, and output it's IPFS Content ID.</p>
+# add-to-web3
 
-! v3 of this action will work with the new w3up api once https://github.com/web3-storage/add-to-web3/issues/87 lands.
+Upload files to [web3.storage](https://web3.storage) from a Github Action, and output the IPFS Content ID.
 
-## Example usage
+A lightweight wrapper around [w3cli](https://github.com/web3-storage/w3cli). As a [composite](https://docs.github.com/en/actions/creating-actions/creating-a-composite-action) github action all it does is configure and call the cli for you. See the steps in [./action.yml](./action.yml).
+
+## Usage
 
 ```yaml
-uses: web3-storage/add-to-web3@v2
-id: web3
+uses: web3-storage/add-to-web3@v3
+id: w3up
 with:
-  web3_token: ${{ secrets.WEB3_STORAGE_TOKEN }}
   path_to_add: 'dist'
+  proof: ${{ secrets.W3_PROOF }}
+  secret_key: ${{ secrets.W3_PRINCIPAL }}
 
+# use the outputs in subsequent steps
 # "bafkreicysg23kiwv34eg2d7qweipxwosdo2py4ldv42nbauguluen5v6am"
-- run: echo ${{ steps.web3.outputs.cid }}
+- run: echo ${{ steps.w3up.outputs.cid }}
+
+# "https://bafkreicysg23kiwv34eg2d7qweipxwosdo2py4ldv42nbauguluen5v6am.ipfs.w3s.link"
+- run: echo ${{ steps.w3up.outputs.url }}
+```
+
+Use [w3cli] to generate a `secret_key` and `proof` to allow this action to upload to a space on.
 
-# "https://dweb.link/ipfs/bafkreicysg23kiwv34eg2d7qweipxwosdo2py4ldv42nbauguluen5v6am"
-- run: echo ${{ steps.web3.outputs.url }}
+Install it from npm and login as described here https://web3.storage/docs/quickstart/ then:
+
+```shell
+# create a signing key for CI.
+# Use the `did` in the input to the next command. 
+# Use `key` as your `secret_key` for add_to_web3.
+$ w3 key create --json
+{
+  "did": "did:key:z6Mk...",
+  "key": "MgCaT7Se2QX9..."
+}
+
+# create a base64 encoded UCAN `proof` 
+# It delegates store and upload permissions to the `did` we created above.
+$ w3 delegation create did:key:z6Mk... -c 'store/add' -c 'upload/add' --base64
+mAYIEAP8OEaJlcm9vdHOAZ3ZlcnNpb24BwwUBcRIg+oHTbzShh1WzBo9ISkonCW+KAcy/+zW8Zb...
 ```
 
+- Use the `key` value from the output of `w3 key create --json` as the `secret_key` for this action.
+- Use the `did value from that command as the audience for `w3 delegation create <audience>` shown above.
+- Use the output of `w3 delegation create <audience>` as the `proof` for this action.
+
+Keep the `secret_key` safe. Save it as a secret on your repo. The `proof` delegates permission from your account to that key to upload to your space. The `proof` can only be used by an agent that holds the `secret_key`.
+
 ## Inputs
 
 ### `path_to_add`
 
 **Required** The path the root directory of your static website or other content that you want to publish to IPFS.
 
-### `web3_token`
+### `secret_key`
 
-**Required** API token for web3.storage
+**Required** The base64 encoded key to use to sign UCAN invocations to web3.storage. 
 
-<details>
-  <summary>Show advanced options: <code>wrap_with_directory</code>, <code>include_hidden</code>, <code>web3_api</code></summary>
-
-### `wrap_with_directory`
+Create one using `w3 key create`. See: https://github.com/web3-storage/w3cli#w3_principal
 
-_Default_ `false`
+### `proof`
 
-Should the `path_to_add` be wrapped in a diretory when creating the IPFS DAG. For most folks using this action the default of `false` is fine. 
+**Required** A base64 encoded UCAN delegating capabilities the signing key above. 
 
-This is the opposite of the default that web3.storage uses, as this action is commonly used to add a directory that contains a static website to IPFS. In that case you want the path_to_add to become the root cid so you can host your site at `https://<cid>.ipfs.dweb.link` rather than `https://<cid>.ipfs.dweb.link/<path_to_add>`.
+Create a proof using w3cli, delegating `store/add' and `upload/add`
 
-If you do want to capture the `path_to_add` path itself in the IPFS DAG then you want to set `wrap_with_directory:true`.
+```shell
+$ AUDIENCE_DID="<the DID for secret_key created by `w3 key create`>"
+$ w3 delegation create $AUDIENCE_DID  -c 'store/add' -c 'upload/add' --base64`
+```
 
-see: https://web3.storage/docs/reference/js-client-library#parameters
+<details>
+  <summary>Show advanced options: <code>hidden</code>, <code>wrap</code></summary>
 
-### `include_hidden`
+### `hidden`
 
 _Default_ `false`
 
 Should hidden files prefixed with a `.` be included when found in the `path_to_add`
 
-see: https://github.com/web3-storage/files-from-path#filesfrompath
+see: See: https://github.com/web3-storage/w3cli#w3-up-path-path
+
+### `wrap`
 
-### `web3_api`
+_Default_ `true`
 
-_Default_ `https://api.web3.storage`
+If `path_to_add` points to a file it will be wrapped in a directory to preserve the filename. To disable that set wrap: "true".
 
-Useful for testing against staging deployments by setting to the api origin of your choice.
+See: https://github.com/web3-storage/w3cli#w3-up-path-path
 
 </details>
 
@@ -72,11 +104,8 @@ e.g. `bafkreicysg23kiwv34eg2d7qweipxwosdo2py4ldv42nbauguluen5v6am`
 The IPFS gateway URL for the directory 
 e.g. `https://dweb.link/ipfs/bafkreicysg23kiwv34eg2d7qweipxwosdo2py4ldv42nbauguluen5v6am`
 
-
 ## Contibuting
 
 💌 Considerate contributions welcome! 
 
-The `dist` folder is commited to the repo as is the curious cultural norm with JS actions, as the repo is the delivery mechanism, so to spare some cycles for the user users, all the deps are bundled into a single /dist/index.js monolith.
-
 <h3 align="center"><a href="https://web3.storage">⁂</a></h3>
diff --git a/action.yml b/action.yml
@@ -4,30 +4,64 @@ author: olizilla
 branding:
   icon: 'box'
   color: 'white'
+
 inputs:
   path_to_add:
     description: 'Directory path to add to IPFS'
     required: true
-  include_hidden:
-    description: 'Include hidden files'
-    required: false
-    default: 'false'
-  web3_token:
-    description: 'API token for web3.storage'
+
+  secret_key:
+    description: 'The base64 encoded key to use to sign UCAN invocations. Create one using `w3 key create`. See: https://github.com/web3-storage/w3cli#w3_principal'
     required: true
-  web3_api:
-    description: 'API URL'
-    required: false
-    default: 'https://api.web3.storage'
-  wrap_with_directory:
-    description: 'Boolean. Default: `false`. Should the path_to_add be wrapped in a diretory when creating the IPFS DAG. For most folks using this, the default of `false` is fine. If you want to add a single file and preserve the filename in the IPFS DAG you may want to set it to `true`.'
+
+  proof:
+    description: 'A base64 encoded UCAN delegating capabilities the secret signing key above. Create using `w3 delegation create --base64`'
+    required: true
+
+  hidden:
+    description: 'Include paths that start with "."'
     required: false
     default: 'false'
+
+  wrap:
+    description: 'If `path_to_add` points to a file it will be wrapped in a directory to preserve the filename. To disable that set wrap: "false"'
+    required: false
+    default: 'true'
+
 outputs:
   cid:
-    description: 'The IPFS Content ID for the directory'
+    description: 'The IPFS Content ID for path_to_add'
+    value: ${{ steps.get_cid.outputs.cid }}
+
   url:
     description: 'The IPFS Gateway URL'
+    value: ${{ steps.get_url.outputs.url }}
+
 runs:
-  using: 'node16'
-  main: 'dist/index.js'
+  using: "composite"
+  steps:
+    - run: npm install -g @web3-storage/w3cli
+      shell: bash
+
+    - run: w3 space add ${{ inputs.proof }}
+      env:
+        W3_PRINCIPAL: ${{ inputs.secret_key }}
+      shell: bash
+
+    - run: w3 up ${{ inputs.path_to_add }} --json --hidden=${{ inputs.hidden }} --wrap=${{ inputs.wrap }} | tee ./w3_up_output.json
+      env:
+        W3_PRINCIPAL: ${{ inputs.secret_key }}
+      shell: bash
+
+    # expect `./w3_up_output.json` to contain "{ "root": { "/":  "bafy..." }}
+    - name: fail if no CID in response
+      run: jq --exit-status --raw-output '.root."/"' ./w3_up_output.json
+      shell: bash
+
+    - id: get_cid
+      run: echo "cid=$(jq --exit-status --raw-output '.root."/"' ./w3_up_output.json)" >> "$GITHUB_OUTPUT"
+      shell: bash
+
+    - id: get_url
+      run: echo "url=https://${{ steps.get_cid.outputs.cid }}.ipfs.w3s.link" >> "$GITHUB_OUTPUT"
+      shell: bash