Merge branch 'release/0.3.0' into stable

Author: holetse

CODE_OF_CONDUCT.md

@@ -55,7 +55,7 @@ further defined and clarified by project maintainers.
 ## Enforcement
 
 Instances of abusive, harassing, or otherwise unacceptable behavior may be
-reported by contacting the project team at [INSERT EMAIL ADDRESS]. All
+reported by contacting the project team at `[email protected]`. All
 complaints will be reviewed and investigated and will result in a response that
 is deemed necessary and appropriate to the circumstances. The project team is
 obligated to maintain confidentiality with regard to the reporter of an incident.

CONTRIBUTING.md

@@ -0,0 +1,68 @@
+# Contributing
+
+You want to contribute? Awesome! We'd love the help. If you have an idea already, great. If not,
+take a look at our [issue tracker][issues] and see if anything appeals. More tests and
+documentation are always appreciated too.
+
+## Getting Started
+
+1. Fork the repository
+2. Make sure the tests pass locally _before_ you start developing.
+3. Write a test or two that cover your feature/bug/refactor (not needed for documentation-only changes)
+4. Make your test pass by adding that slick new code.
+5. Add documentation for your change (if appropriate)
+6. Run `mix credo --strict` and `mix dialyzer` to ensure you haven't missed any coding standards.
+7. Commit your changes and open a [pull request][pulls].
+8. Wait for a speedy review of your change! Thank you!
+
+## Development Dependencies
+
+In order to run the functional tests, you need to have [Docker][docker] installed locally. The
+community edition is fine, but you'll want to avoid the old versions that require a VM.
+
+## Code Standards
+
+Most of the code standards can be found in `.credo.exs`, and will be checked automatically by the
+CI process. When in doubt, follow the standards in the file you are changing. Terse but descriptive
+variable and function names make us happy. The standard Elixir guide on [writing documentation][writing-docs]
+has some good tips on names. Documentation for new public functions is expected, as are tests for
+any code change.
+
+Good commit messages and PR descriptions are also important. See our guide on
+[commit messages](https://github.com/labzero/guides/blob/master/process/commit_guide.md) for more details.
+
+## Testing
+
+Good tests are arguably more important than good code, so please take a moment to make sure
+you have a few with your PR. Try to avoid mock-only tests, as they can get out of sync with reality
+fairly easily. They are great for doing basic unit testing though! You'll see we use
+[mock](https://github.com/jjh42/mock) as our mocking framework of choice.
+
+Functional tests are much more reliable with a tool like Bootleg, and there are plenty of examples
+in the project. `Bootleg.FunctionalCase` provides a simple interface for writing [Docker][docker]
+based functional tests. By default each test case will get a single docker container provisioned,
+and the details will be passed to `setup` under the key `hosts`. You can request more containers
+using `@tag boot: 2` where `2` is the number of containers you'd like. During test development it's
+often helpful to have the containers left running after the tests finish, and you can request that
+by setting the `ENV` variable `TEST_LEAVE_CONTAINER` when running your tests. It's best to limit how
+many tests are run in that case, or you may kill your machine with too many docker containers at once.
+
+If you need a project to test against (this a deployment tool after all), take a look at
+`Bootleg.Fixtures.inflate_project/1`. It will take any of the fixture projects and create a new
+instance for use during testing. The `test/fixtures` directory contains all the currently available
+fixture projects. Instances of projects created via `inflate_project/1` will be cleaned up when the
+test suite exits, but you can suppress that by setting `TEST_LEAVE_TEMP` in the `ENV`. Fixtures are
+always inflated to your OS temporary directory.
+
+## Contact
+
+You can reach the core Bootleg team in [#deployment](https://elixir-lang.slack.com/messages/C0LH49EPQ)
+or [#bootleg](https://elixir-lang.slack.com/messages/C6D2BQY4R) on Elixir Slack. We are also reachable
+via email at `[email protected]`. Don't hesitate to get in touch, we'd love to hear from you.
+
+Use the [issue tracker][issues] for bug reports or feature requests.
+
+  [issues]: https://github.com/labzero/bootleg/issues
+  [pulls]: https://github.com/labzero/bootleg/pulls
+  [writing-docs]: http://elixir-lang.org/docs/stable/elixir/writing-documentation.html
+  [docker]: https://www.docker.com/

README.md

@@ -15,7 +15,7 @@ add additional support.
 ```elixir
 def deps do
   [{:distillery, "~> 1.3",
-   {:bootleg, "~> 0.1.0"}]
+   {:bootleg, "~> 0.3"}]
 end
 ```
 
@@ -151,6 +151,9 @@ Alternatively the above commands can be rolled into one with:
 mix bootleg.update production
 ```
 
+Note that `bootleg.update` will stop any running nodes and then perform a cold start. The stop is performed with
+the task `stop_silent`, which differs from `stop` in that it does not fail if the node is already stopped.
+
 ## Admin Commands
 
 Bootleg has a set of commands to check up on your running nodes:
@@ -162,6 +165,15 @@ mix bootleg.stop production      # Stops a deployed release.
 mix bootleg.ping production      # Check status of running nodes
 ```
 
+## Other Comamnds
+
+Bootleg has a few utility commands to help streamline its usage:
+
+```console
+mix bootleg.init             # Initializes a project for use with Bootleg
+mix bootleg.invoke <task>    # Calls an arbitrary Bootleg task
+```
+
 ## Hooks
 
 Hooks may be defined by the user in order to perform additional (or exceptional)
@@ -287,7 +299,8 @@ end
 
 The workhorse of the Bootleg DSL is `remote`: it executes shell commands on remote servers and returns
 the results. It takes a role and a block of commands to execute. The commands are executed on all servers
-belonging to the role, and raises an `SSHError` if an error is encountered.
+belonging to the role, and raises an `SSHError` if an error is encountered. Optionally, a list of options
+can be provided to filter the hosts where the commands are run.
 
 ```elixir
 use Bootleg.Config
@@ -312,6 +325,11 @@ end
 remote :app do
   "false"
 end
+
+# filtering - only runs on app hosts with an option of primary set to true
+remote :app, primary: true do
+  "mix ecto.migrate"
+end
 ```
 
 ## Phoenix Support
@@ -326,8 +344,8 @@ for building phoenix releases.
 # mix.exs
 def deps do
   [{:distillery, "~> 1.3"},
-  {:bootleg, "~> 0.2.0"},
-  {:bootleg_phoenix, "~> 0.1.0"}]
+  {:bootleg, "~> 0.3"},
+  {:bootleg_phoenix, "~> 0.1"}]
 end
 ```
 

lib/bootleg/config.ex

@@ -9,7 +9,7 @@ defmodule Bootleg.Config do
   defmacro __using__(_) do
     quote do
       import Bootleg.Config, only: [role: 2, role: 3, config: 2, config: 0, before_task: 2,
-        after_task: 2, invoke: 1, task: 2, remote: 1, remote: 2, load: 1]
+        after_task: 2, invoke: 1, task: 2, remote: 1, remote: 2, remote: 3, load: 1, upload: 3]
     end
   end
 
@@ -22,7 +22,7 @@ defmodule Bootleg.Config do
 
   `name` is the name of the role, and is globally unique. Calling `role/3` multiple times with
   the same name will result in the host lists being merged. If the same host shows up mutliple
-  times, it will have its `options` merged.
+  times, it will have its `options` merged. The name `:all` is reserved and cannot be used here.
 
   `hosts` can be a single hostname, or a `List` of hostnames.
 
@@ -41,6 +41,9 @@ defmodule Bootleg.Config do
   """
   defmacro role(name, hosts, options \\ []) do
     # user is in the role options for scm
+    if name == :all do
+      raise ArgumentError, ":all is reserved by bootleg and refers to all defined roles."
+    end
     user = Keyword.get(options, :user, System.get_env("USER"))
     ssh_options = Enum.filter(options, &Enum.member?(SSH.supported_options, elem(&1, 0)) == true)
     role_options =
@@ -325,11 +328,28 @@ defmodule Bootleg.Config do
   end
 
   defmacro remote(role, do: {:__block__, _, lines}) do
-    quote do: remote(unquote(role), unquote(lines))
+    quote do: remote(unquote(role), [], unquote(lines))
   end
 
   defmacro remote(role, do: lines) do
-    quote do: remote(unquote(role), unquote(lines))
+    quote do: remote(unquote(role), [], unquote(lines))
+  end
+
+  @doc """
+  Executes commands on all remote hosts within a role.
+
+  This is equivalent to calling `remote/3` with a `filter` of `[]`.
+  """
+  defmacro remote(role, lines) do
+    quote do: remote(unquote(role), [], unquote(lines))
+  end
+
+  defmacro remote(role, filter, do: {:__block__, _, lines}) do
+    quote do: remote(unquote(role), unquote(filter), unquote(lines))
+  end
+
+  defmacro remote(role, filter, do: lines) do
+    quote do: remote(unquote(role), unquote(filter), unquote(lines))
   end
 
   @doc """
@@ -343,6 +363,10 @@ defmodule Bootleg.Config do
   used as a command. Each command will be simulataneously executed on all hosts in the role. Once
   all hosts have finished executing the command, the next command in the list will be sent.
 
+  `filter` is an optional `Keyword` list of host options to filter with. Any host whose options match
+  the filter will be included in the remote execution. A host matches if it has all of the filtering
+  options defined and the values match (via `==/2`) the filter.
+
   `role` can be a single role, a list of roles, or the special role `:all` (all roles). If the same host
   exists in multiple roles, the commands will be run once for each role where the host shows up. In the
   case of multiple roles, each role is processed sequentially.
@@ -367,18 +391,22 @@ defmodule Bootleg.Config do
 
   # runs for hosts found in :build first, then for hosts in :app
   remote [:build, :app], do: "hostname"
+
+  # only runs on `host1.example.com`
+  role :build, "host2.example.com"
+  role :build, "host1.example.com", primary: true, another_attr: :cat
+
+  remote :build, primary: true do
+    "hostname"
+  end
   ```
   """
-  defmacro remote(role, lines) do
-    roles = if role == :all do
-      quote do: Keyword.keys(Bootleg.Config.Agent.get(:roles))
-    else
-      quote do: List.wrap(unquote(role))
-    end
+  defmacro remote(role, filter, lines) do
+    roles = unpack_role(role)
     quote bind_quoted: binding() do
       Enum.reduce(roles, [], fn role, outputs ->
         role
-        |> SSH.init
+        |> SSH.init([], filter)
         |> SSH.run!(lines)
         |> SSH.merge_run_results(outputs)
       end)
@@ -401,6 +429,56 @@ defmodule Bootleg.Config do
     end
   end
 
+  @doc """
+  Uploads a local file to remote hosts.
+
+  Uploading works much like `remote/3`, but instead of transferring shell commands over SSH,
+  it transfers files via SCP. The remote host does need to support SCP, which should be provided
+  by most SSH implementations automatically.
+
+  `role` can either be a single role name, a list of roles, or a list of roles and filter
+  attributes. The special `:all` role is also supported. See `remote/3` for details.
+
+  `local_path` can either be a file or directory found on the local machine. If its a directory,
+  the entire directory will be recursively copied to the remote hosts. Relative paths are resolved
+  relative to the root of the local project.
+
+  `remote_path` is the file or directory where the transfered files should be placed. The semantics
+  of how `remote_path` is treated vary depending on what `local_path` refers to. If `local_path` points
+  to a file, `remote_path` is treated as a file unless it's `.` or ends in `/`, in which case it's
+  treated as a directory and the filename of the local file will be used. If `local_path` is a directory,
+  `remote_path` is treated as a directory as well. Relative paths are resolved relative to the projects
+  remote `workspace`. Missing directories are not implicilty created.
+
+  The files on the remote server are created using the authenticating user's `uid`/`gid` and `umask`.
+
+  ```
+  use Bootleg.Config
+
+  # copies ./my_file to ./new_name on the remote host
+  upload :app, "my_file", "new_name"
+
+  # copies ./my_file to ./a_dir/my_file on the remote host. ./a_dir must already exist
+  upload :app, "my_file", "a_dir/"
+
+  # recursively copies ./some_dir to ./new_dir on the remote host. ./new_dir will be created if missing
+  upload :app, "some_dir", "new_dir"
+
+  # copies ./my_file to /tmp/foo on the remote host
+  upload :app, "my_file", "/tmp/foo"
+  """
+  defmacro upload(role, local_path, remote_path) do
+    {roles, filters} = split_roles_and_filters(role)
+    roles = unpack_role(roles)
+    quote bind_quoted: binding() do
+      Enum.each(roles, fn role ->
+        role
+        |> SSH.init([], filters)
+        |> SSH.upload(local_path, remote_path)
+      end)
+    end
+  end
+
   @doc false
   @spec get_config(atom, any) :: any
   def get_config(key, default \\ nil) do
@@ -418,4 +496,23 @@ defmodule Bootleg.Config do
   def version do
     get_config(:version, Project.config[:version])
   end
+
+  @doc false
+  @spec split_roles_and_filters(atom | keyword) :: {[atom], keyword}
+  defp split_roles_and_filters(role) do
+    role
+    |> List.wrap
+    |> Enum.split_while(fn term -> !is_tuple(term) end)
+  end
+
+  @doc false
+  @spec unpack_role(atom | keyword) :: tuple
+  defp unpack_role(role) do
+    wrapped_role = List.wrap(role)
+    if Enum.any?(wrapped_role, fn role -> role == :all end) do
+      quote do: Keyword.keys(Bootleg.Config.Agent.get(:roles))
+    else
+      quote do: unquote(wrapped_role)
+    end
+  end
 end