Prep 1.5.0 release (#2098)

* Updating the Changelog and Upgrade guide for 1.5.0 release.

Signed-off-by: Ed Welch <[email protected]>

* Changing release number in all the docs

Signed-off-by: Ed Welch <[email protected]>

README.md

@@ -29,11 +29,9 @@ Loki differs from Prometheus by focusing on logs instead of metrics, and deliver
 
 ## Getting started
 
-* [Installing Loki](https://github.com/grafana/loki/tree/v1.4.1/docs/installation/README.md)
-* [Installing
-Promtail](https://github.com/grafana/loki/tree/v1.4.1/docs/clients/promtail/installation.md)
-* [Getting
-Started Guide](https://github.com/grafana/loki/tree/v1.4.1/docs/getting-started/README.md)
+* [Installing Loki](https://github.com/grafana/loki/tree/v1.5.0/docs/installation/README.md)
+* [Installing Promtail](https://github.com/grafana/loki/tree/v1.5.0/docs/clients/promtail/installation.md)
+* [Getting Started Guide](https://github.com/grafana/loki/tree/v1.5.0/docs/getting-started/README.md)
 
 ## Upgrading
 
@@ -42,25 +40,26 @@ Started Guide](https://github.com/grafana/loki/tree/v1.4.1/docs/getting-started/
 ### Documentation
 
 * [master](./docs/README.md)
+* [v1.5.0](https://github.com/grafana/loki/tree/v1.5.0/docs/README.md)
 * [v1.4.1](https://github.com/grafana/loki/tree/v1.4.1/docs/README.md)
 * [v1.4.0](https://github.com/grafana/loki/tree/v1.4.0/docs/README.md)
 * [v1.3.0](https://github.com/grafana/loki/tree/v1.3.0/docs/README.md)
 * [v1.2.0](https://github.com/grafana/loki/tree/v1.2.0/docs/README.md)
 * [v1.1.0](https://github.com/grafana/loki/tree/v1.1.0/docs/README.md)
 * [v1.0.0](https://github.com/grafana/loki/tree/v1.0.0/docs/README.md)
 
-Commonly used sections (from the latest release v1.4.1):
+Commonly used sections (from the latest release v1.5.0):
 
-- [API documentation](https://github.com/grafana/loki/tree/v1.4.1/docs/api.md) for alternative ways of getting logs into Loki.
+- [API documentation](https://github.com/grafana/loki/tree/v1.5.0/docs/api.md) for alternative ways of getting logs into Loki.
 - [Labels](https://github.com/grafana/loki/blob/master/docs/getting-started/labels.md)
-- [Operations](https://github.com/grafana/loki/tree/v1.4.1/docs/operations) for important aspects of running Loki.
-- [Promtail](https://github.com/grafana/loki/tree/v1.4.1/docs/clients/promtail) is an agent which can tail your log files and push them to Loki.
-- [Pipelines](https://github.com/grafana/loki/tree/v1.4.1/docs/clients/promtail/pipelines.md) for detailed log processing pipeline documentation
-- [Docker Logging Driver](https://github.com/grafana/loki/tree/v1.4.1/docs/clients/docker-driver) is a docker plugin to send logs directly to Loki from Docker containers.
-- [LogCLI](https://github.com/grafana/loki/tree/v1.4.1/docs/getting-started/logcli.md) on how to query your logs without Grafana.
-- [Loki Canary](https://github.com/grafana/loki/tree/v1.4.1/docs/operations/loki-canary.md) for monitoring your Loki installation for missing logs.
-- [Troubleshooting](https://github.com/grafana/loki/tree/v1.4.1/docs/getting-started/troubleshooting.md) for help around frequent error messages.
-- [Loki in Grafana](https://github.com/grafana/loki/tree/v1.4.1/docs/getting-started/grafana.md) for how to set up a Loki datasource in Grafana and query your logs.
+- [Operations](https://github.com/grafana/loki/tree/v1.5.0/docs/operations) for important aspects of running Loki.
+- [Promtail](https://github.com/grafana/loki/tree/v1.5.0/docs/clients/promtail) is an agent which can tail your log files and push them to Loki.
+- [Pipelines](https://github.com/grafana/loki/tree/v1.5.0/docs/clients/promtail/pipelines.md) for detailed log processing pipeline documentation
+- [Docker Logging Driver](https://github.com/grafana/loki/tree/v1.5.0/docs/clients/docker-driver) is a docker plugin to send logs directly to Loki from Docker containers.
+- [LogCLI](https://github.com/grafana/loki/tree/v1.5.0/docs/getting-started/logcli.md) on how to query your logs without Grafana.
+- [Loki Canary](https://github.com/grafana/loki/tree/v1.5.0/docs/operations/loki-canary.md) for monitoring your Loki installation for missing logs.
+- [Troubleshooting](https://github.com/grafana/loki/tree/v1.5.0/docs/getting-started/troubleshooting.md) for help around frequent error messages.
+- [Loki in Grafana](https://github.com/grafana/loki/tree/v1.5.0/docs/getting-started/grafana.md) for how to set up a Loki datasource in Grafana and query your logs.
 
 ## Getting Help
 

docs/clients/promtail/installation.md

@@ -12,7 +12,7 @@ Every release includes binaries for Promtail which can be found on the
 
 ```bash
 # modify tag to most recent version
-$ docker pull grafana/promtail:1.4.1
+$ docker pull grafana/promtail:1.5.0
 ```
 
 ## Helm

docs/installation/docker.md

@@ -15,10 +15,10 @@ For production, we recommend Tanka or Helm.
 Copy and paste the commands below into your command line.
 
 ```bash
-wget https://raw.githubusercontent.com/grafana/loki/v1.4.1/cmd/loki/loki-local-config.yaml -O loki-config.yaml
-docker run -v $(pwd):/mnt/config -p 3100:3100 grafana/loki:1.4.1 -config.file=/mnt/config/loki-config.yaml
-wget https://raw.githubusercontent.com/grafana/loki/v1.4.1/cmd/promtail/promtail-docker-config.yaml -O promtail-config.yaml
-docker run -v $(pwd):/mnt/config -v /var/log:/var/log grafana/promtail:1.4.1 -config.file=/mnt/config/promtail-config.yaml
+wget https://raw.githubusercontent.com/grafana/loki/v1.5.0/cmd/loki/loki-local-config.yaml -O loki-config.yaml
+docker run -v $(pwd):/mnt/config -p 3100:3100 grafana/loki:1.5.0 -config.file=/mnt/config/loki-config.yaml
+wget https://raw.githubusercontent.com/grafana/loki/v1.5.0/cmd/promtail/promtail-docker-config.yaml -O promtail-config.yaml
+docker run -v $(pwd):/mnt/config -v /var/log:/var/log grafana/promtail:1.5.0 -config.file=/mnt/config/promtail-config.yaml
 ```
 
 When finished, loki-config.yaml and promtail-config.yaml are downloaded in the directory you chose. Docker containers are running Loki and Promtail using those config files.
@@ -31,10 +31,10 @@ Copy and paste the commands below into your terminal. Note that you will need to
 
 ```bash
 cd "<local-path>"
-wget https://raw.githubusercontent.com/grafana/loki/v1.4.1/cmd/loki/loki-local-config.yaml -O loki-config.yaml
-docker run -v <local-path>:/mnt/config -p 3100:3100 grafana/loki:1.4.1 --config.file=/mnt/config/loki-config.yaml
-wget https://raw.githubusercontent.com/grafana/loki/v1.4.1/cmd/promtail/promtail-docker-config.yaml -O promtail-config.yaml
-docker run -v <local-path>:/mnt/config -v /var/log:/var/log grafana/promtail:1.4.1 --config.file=/mnt/config/promtail-config.yaml
+wget https://raw.githubusercontent.com/grafana/loki/v1.5.0/cmd/loki/loki-local-config.yaml -O loki-config.yaml
+docker run -v <local-path>:/mnt/config -p 3100:3100 grafana/loki:1.5.0 --config.file=/mnt/config/loki-config.yaml
+wget https://raw.githubusercontent.com/grafana/loki/v1.5.0/cmd/promtail/promtail-docker-config.yaml -O promtail-config.yaml
+docker run -v <local-path>:/mnt/config -v /var/log:/var/log grafana/promtail:1.5.0 --config.file=/mnt/config/promtail-config.yaml
 ```
 
 When finished, loki-config.yaml and promtail-config.yaml are downloaded in the directory you chose. Docker containers are running Loki and Promtail using those config files.
@@ -44,6 +44,6 @@ Navigate to http://localhost:3100/metrics to view the output.
 ## Install with Docker Compose
 
 ```bash
-$ wget https://raw.githubusercontent.com/grafana/loki/v1.4.1/production/docker-compose.yaml -O docker-compose.yaml
+$ wget https://raw.githubusercontent.com/grafana/loki/v1.5.0/production/docker-compose.yaml -O docker-compose.yaml
 $ docker-compose -f docker-compose.yaml up
 ```

docs/operations/loki-canary.md

@@ -67,7 +67,7 @@ Loki Canary is also provided as a Docker container image:
 
 ```bash
 # change tag to the most recent release
-$ docker pull grafana/loki-canary:1.4.1
+$ docker pull grafana/loki-canary:1.5.0
 ```
 
 ### Kubernetes

docs/operations/upgrade.md

@@ -8,6 +8,10 @@ On this page we will document any upgrade issues/gotchas/considerations we are a
 
 ## 1.5.0
 
+Note: The required upgrade path outlined for version 1.4.0 below is still true for moving to 1.5.0 from any release older than 1.4.0 (e.g. 1.3.0->1.5.0 needs to also look at the 1.4.0 upgrade requirements).
+
+### Breaking config changes!
+
 Loki 1.5.0 vendors Cortex v1.0.0 (congratulations!), which has a [massive list of changes](https://cortexmetrics.io/docs/changelog/#1-0-0-2020-04-02).
 
 While changes in the command line flags affect Loki as well, we usually recommend people to use configuration file instead.
@@ -16,6 +20,92 @@ Cortex has done lot of cleanup in the configuration files, and you are strongly
 
 Following fields were removed from YAML configuration completely: `claim_on_rollout` (always true), `normalise_tokens` (always true).
 
+#### Test Your Config
+
+To see if your config needs to change, one way to quickly test is to download a 1.5.0 (or newer) binary from the [release page](https://github.com/grafana/loki/releases/tag/v1.5.0)
+
+Then run the binary providing your config file `./loki-linux-amd64 -config.file=myconfig.yaml`
+
+If there are configs which are no longer valid you will see errors immediately:
+
+```shell
+./loki-linux-amd64 -config.file=loki-local-config.yaml
+failed parsing config: loki-local-config.yaml: yaml: unmarshal errors:
+  line 35: field dynamodbconfig not found in type aws.StorageConfig
+```
+
+Referencing the [list of diffs](https://cortexmetrics.io/docs/changelog/#config-file-breaking-changes) I can see this config changed:
+
+```diff
+-  dynamodbconfig:
++  dynamodb:
+```
+
+Also several other AWS related configs changed and would need to udpate those as well.
+
+
+### Loki Docker Image User and File Location Changes
+
+To improve security concerns, in 1.5.0 the Docker container no longer runs the loki process as `root` and instead the process runs as user `loki` with UID `10001` and GID `10001`
+
+This may affect people in a couple ways:
+
+#### Loki Port
+
+If you are running Loki with a config that opens a port number above 1000 (which is the default, 3100 for HTTP and 9095 for GRPC) everything should work fine in regards to ports.
+
+If you are running Loki with a config that opens a port number less than 1000 Linux normally requires root permissions to do this, HOWEVER in the Docker container we run `setcap cap_net_bind_service=+ep /usr/bin/loki`
+
+This capability lets the loki process bind to a port less than 1000 when run as a non root user.
+
+Not every environment will allow this capability however, it's possible to restrict this capability in linux.  If this restriction is in place, you will be forced to run Loki with a config that has HTTP and GRPC ports above 1000.
+
+#### Filesystem
+
+**Please note the location Loki is looking for files with the provided config in the docker image has changed**
+
+In 1.4.0 and earlier the included config file in the docker container was using directories:
+
+```
+/tmp/loki/index
+/tmp/loki/chunks
+```
+
+In 1.5.0 this has changed:
+
+```
+/loki/index
+/loki/chunks
+```
+
+This will mostly affect anyone using docker-compose or docker to run Loki and are specifying a volume to persist storage.
+
+**There are two concerns to track here, one is the correct ownership of the files and the other is making sure your mounts updated to the new location.**
+
+One possible upgrade path would look like this:
+
+If I were running Loki with this command `docker run -d --name=loki --mount source=loki-data,target=/tmp/loki -p 3100:3100 grafana/loki:1.4.0`
+
+This would mount a docker volume named `loki-data` to the `/temp/loki` folder which is where Loki will persist the `index` and `chunks` folder in 1.4.0
+
+To move to 1.5.0 I can do the following (please note that your container names and paths and volumes etc may be different):
+
+```
+docker stop loki
+docker rm loki
+docker run --rm --name="loki-perm" -it --mount source=loki-data,target=/mnt ubuntu /bin/bash
+cd /mnt
+chown -R 10001:10001 ./*
+exit
+docker run -d --name=loki --mount source=loki-data,target=/loki -p 3100:3100 grafana/loki:1.5.0
+```
+
+Notice the change in the `target=/loki` for 1.5.0 to the new data directory location specified in the [included Loki config file](../../cmd/loki/loki-docker-config.yaml).
+
+The intermediate step of using an ubuntu image to change the ownership of the Loki files to the new user might not be necessary if you can easily access these files to run the `chown` command directly.  
+That is if you have access to `/var/lib/docker/volumes` or if you mounted to a different local filesystem directory, you can change the ownership directly without using a container. 
+
+
 ## 1.4.0
 
 Loki 1.4.0 vendors Cortex v0.7.0-rc.0 which contains [several breaking config changes](https://github.com/cortexproject/cortex/blob/v0.7.0-rc.0/CHANGELOG.md).
@@ -84,4 +174,3 @@ If you attempt to add a v1.4.0 ingester to a ring created by Loki v1.2.0 or olde
 This will result in distributors failing to write and a general ingestion failure for the system.
 
 If this happens to you, you will want to rollback your deployment immediately. You need to remove the v1.4.0 ingester from the ring ASAP, this should allow the existing ingesters to re-insert their tokens.  You will also want to remove any v1.4.0 distributors as they will not understand the old ring either and will fail to send traffic.
-

production/docker-compose.yaml

@@ -5,15 +5,15 @@ networks:
 
 services:
   loki:
-    image: grafana/loki:1.4.1
+    image: grafana/loki:1.5.0
     ports:
       - "3100:3100"
     command: -config.file=/etc/loki/local-config.yaml
     networks:
       - loki
 
   promtail:
-    image: grafana/promtail:1.4.1
+    image: grafana/promtail:1.5.0
     volumes:
       - /var/log:/var/log
     command: -config.file=/etc/promtail/docker-config.yaml