Skip to content

Commit

Permalink
Merge pull request #122 from miccehedin/app-engine
Browse files Browse the repository at this point in the history 
App engine
  • Loading branch information
@marickvantuil
marickvantuil authored Feb 13, 2024
2 parents bf14e38 + a4a3098 commit e5a6cb3
Show file tree
Hide file tree
Showing 12 changed files with 278 additions and 158 deletions.
28 changes: 21 additions & 7 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,16 @@ Please check the [Laravel support policy](https://laravel.com/docs/master/releas
'queue' => env('STACKKIT_CLOUD_TASKS_QUEUE', 'default'),
'service_account_email' => env('STACKKIT_CLOUD_TASKS_SERVICE_EMAIL', ''),
'signed_audience' => env('STACKKIT_CLOUD_TASKS_SIGNED_AUDIENCE', true),

// Required when using AppEngine
'app_engine' => env('STACKKIT_APP_ENGINE_TASK', false),
'app_engine_service' => env('STACKKIT_APP_ENGINE_SERVICE', ''),

// Required when not using AppEngine
'handler' => env('STACKKIT_CLOUD_TASKS_HANDLER', ''),
'service_account_email' => env('STACKKIT_CLOUD_TASKS_SERVICE_EMAIL', ''),
'signed_audience' => env('STACKKIT_CLOUD_TASKS_SIGNED_AUDIENCE', true),

// Optional: The deadline in seconds for requests sent to the worker. If the worker
// does not respond by this deadline then the request is cancelled and the attempt
// is marked as a DEADLINE_EXCEEDED failure.
Expand All @@ -64,13 +74,17 @@ Now that the package is installed, the final step is to set the correct environm

Please check the table below on what the values mean and what their value should be.

| Environment variable | Description |Example
--------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---
| `STACKKIT_CLOUD_TASKS_PROJECT` | The project your queue belongs to. |`my-project`
| `STACKKIT_CLOUD_TASKS_LOCATION` | The region where the project is hosted |`europe-west6`
| `STACKKIT_CLOUD_TASKS_QUEUE` | The default queue a job will be added to |`emails`
| `STACKKIT_CLOUD_TASKS_SERVICE_EMAIL` | The email address of the service account. Important, it should have the correct roles. See the section below which roles. |`[email protected]`
| `STACKKIT_CLOUD_TASKS_HANDLER` (optional) | The URL that Cloud Tasks will call to process a job. This should be the URL to your Laravel app. By default we will use the URL that dispatched the job. |`https://<your website>.com`
| Environment variable | Description |Example
---------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------|---
| `STACKKIT_CLOUD_TASKS_PROJECT` | The project your queue belongs to. |`my-project`
| `STACKKIT_CLOUD_TASKS_LOCATION` | The region where the project is hosted. |`europe-west6`
| `STACKKIT_CLOUD_TASKS_QUEUE` | The default queue a job will be added to. |`emails`
| **App Engine**
| `STACKKIT_APP_ENGINE_TASK` (optional) | Set to true to use App Engine task (else a Http task will be used). Defaults to false. |`true`
| `STACKKIT_APP_ENGINE_SERVICE` (optional) | The App Engine service to handle the task (only if using App Engine task). |`api`
| **Non- App Engine apps**
| `STACKKIT_CLOUD_TASKS_SERVICE_EMAIL` (optional) | The email address of the service account. Important, it should have the correct roles. See the section below which roles. |`[email protected]`
| `STACKKIT_CLOUD_TASKS_HANDLER` (optional) | The URL that Cloud Tasks will call to process a job. This should be the URL to your Laravel app. By default we will use the URL that dispatched the job. |`https://<your website>.com`
| `STACKKIT_CLOUD_TASKS_SIGNED_AUDIENCE` (optional) | True or false depending if you want extra security by signing the audience of your tasks. May misbehave in certain Cloud Run setups. Defaults to true. | `true`
</details>
<details>
Expand Down
2 changes: 0 additions & 2 deletions src/CloudTasksConnector.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,6 @@ class CloudTasksConnector implements ConnectorInterface
{
public function connect(array $config)
{
Config::validate($config);

// The handler is the URL which Cloud Tasks will call with the job payload. This
// URL of the handler can be manually set through an environment variable, but
// if it is not then we will choose a sensible default (the current app url)
Expand Down
93 changes: 57 additions & 36 deletions src/CloudTasksQueue.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

namespace Stackkit\LaravelGoogleCloudTasksQueue;

use Google\Cloud\Tasks\V2\AppEngineHttpRequest;
use Google\Cloud\Tasks\V2\AppEngineRouting;
use Google\Cloud\Tasks\V2\CloudTasksClient;
use Google\Cloud\Tasks\V2\HttpMethod;
use Google\Cloud\Tasks\V2\HttpRequest;
Expand All @@ -14,6 +16,7 @@
use Illuminate\Support\Carbon;
use Illuminate\Support\Str;
use Stackkit\LaravelGoogleCloudTasksQueue\Events\TaskCreated;

use function Safe\json_decode;
use function Safe\json_encode;

Expand All @@ -36,7 +39,7 @@ public function __construct(array $config, CloudTasksClient $client, $dispatchAf
/**
* Get the size of the queue.
*
* @param string|null $queue
* @param string|null $queue
* @return int
*/
public function size($queue = null)
Expand All @@ -48,11 +51,11 @@ public function size($queue = null)
/**
* Fallback method for Laravel 6x and 7x
*
* @param \Closure|string|object $job
* @param string $payload
* @param string $queue
* @param \DateTimeInterface|\DateInterval|int|null $delay
* @param callable $callback
* @param \Closure|string|object $job
* @param string $payload
* @param string $queue
* @param \DateTimeInterface|\DateInterval|int|null $delay
* @param callable $callback
* @return mixed
*/
protected function enqueueUsing($job, $payload, $queue, $delay, $callback)
Expand All @@ -67,9 +70,9 @@ protected function enqueueUsing($job, $payload, $queue, $delay, $callback)
/**
* Push a new job onto the queue.
*
* @param string|object $job
* @param mixed $data
* @param string|null $queue
* @param string|object $job
* @param mixed $data
* @param string|null $queue
* @return void
*/
public function push($job, $data = '', $queue = null)
Expand All @@ -88,25 +91,25 @@ function ($payload, $queue) {
/**
* Push a raw payload onto the queue.
*
* @param string $payload
* @param string|null $queue
* @param array $options
* @param string $payload
* @param string|null $queue
* @param array $options
* @return string
*/
public function pushRaw($payload, $queue = null, array $options = [])
{
$delay = ! empty($options['delay']) ? $options['delay'] : 0;
$delay = !empty($options['delay']) ? $options['delay'] : 0;

return $this->pushToCloudTasks($queue, $payload, $delay);
}

/**
* Push a new job onto the queue after a delay.
*
* @param \DateTimeInterface|\DateInterval|int $delay
* @param string|object $job
* @param mixed $data
* @param string|null $queue
* @param \DateTimeInterface|\DateInterval|int $delay
* @param string|object $job
* @param mixed $data
* @param string|null $queue
* @return void
*/
public function later($delay, $job, $data = '', $queue = null)
Expand All @@ -125,8 +128,8 @@ function ($payload, $queue, $delay) {
/**
* Push a job to Cloud Tasks.
*
* @param string|null $queue
* @param string $payload
* @param string|null $queue
* @param string $payload
* @param \DateTimeInterface|\DateInterval|int $delay
* @return string
*/
Expand All @@ -136,10 +139,6 @@ protected function pushToCloudTasks($queue, $payload, $delay = 0)
$queueName = $this->client->queueName($this->config['project'], $this->config['location'], $queue);
$availableAt = $this->availableAt($delay);

$httpRequest = $this->createHttpRequest();
$httpRequest->setUrl($this->getHandler());
$httpRequest->setHttpMethod(HttpMethod::POST);

$payload = json_decode($payload, true);

// Laravel 7+ jobs have a uuid, but Laravel 6 doesn't have it.
Expand All @@ -152,11 +151,38 @@ protected function pushToCloudTasks($queue, $payload, $delay = 0)
// value and need to manually set and update the number of times a task has been attempted.
$payload = $this->withAttempts($payload);

$httpRequest->setBody(json_encode($payload));

$task = $this->createTask();
$task->setName($this->taskName($queue, $payload));
$task->setHttpRequest($httpRequest);

if (!empty($this->config['app_engine'])) {
$path = \Safe\parse_url(route('cloud-tasks.handle-task'), PHP_URL_PATH);

$appEngineRequest = new AppEngineHttpRequest();
$appEngineRequest->setRelativeUri($path);
$appEngineRequest->setHttpMethod(HttpMethod::POST);
$appEngineRequest->setBody(json_encode($payload));
if (!empty($service = $this->config['app_engine_service'])) {
$routing = new AppEngineRouting();
$routing->setService($service);
$appEngineRequest->setAppEngineRouting($routing);
}
$task->setAppEngineHttpRequest($appEngineRequest);
} else {
$httpRequest = $this->createHttpRequest();
$httpRequest->setUrl($this->getHandler());
$httpRequest->setHttpMethod(HttpMethod::POST);

$httpRequest->setBody(json_encode($payload));

$token = new OidcToken;
$token->setServiceAccountEmail($this->config['service_account_email']);
if ($audience = $this->getAudience()) {
$token->setAudience($audience);
}
$httpRequest->setOidcToken($token);
$task->setHttpRequest($httpRequest);
}


// The deadline for requests sent to the app. If the app does not respond by
// this deadline then the request is cancelled and the attempt is marked as
Expand All @@ -165,13 +191,6 @@ protected function pushToCloudTasks($queue, $payload, $delay = 0)
$task->setDispatchDeadline(new Duration(['seconds' => $this->config['dispatch_deadline']]));
}

$token = new OidcToken;
$token->setServiceAccountEmail($this->config['service_account_email']);
if ($audience = $this->getAudience()) {
$token->setAudience($audience);
}
$httpRequest->setOidcToken($token);

if ($availableAt > time()) {
$task->setScheduleTime(new Timestamp(['seconds' => $availableAt]));
}
Expand All @@ -186,7 +205,7 @@ protected function pushToCloudTasks($queue, $payload, $delay = 0)
private function withUuid(array $payload): array
{
if (!isset($payload['uuid'])) {
$payload['uuid'] = (string) Str::uuid();
$payload['uuid'] = (string)Str::uuid();
}

return $payload;
Expand Down Expand Up @@ -227,7 +246,7 @@ private function withAttempts(array $payload): array
/**
* Pop the next job off of the queue.
*
* @param string|null $queue
* @param string|null $queue
* @return \Illuminate\Contracts\Queue\Job|null
*/
public function pop($queue = null)
Expand All @@ -251,11 +270,13 @@ public function delete(CloudTasksJob $job): void

$queue = $job->getQueue() ?: $this->config['queue']; // @todo: make this a helper method somewhere.

$headerTaskName = request()->headers->get('X-Cloudtasks-Taskname')
?? request()->headers->get('X-AppEngine-TaskName');
$taskName = $this->client->taskName(
$config['project'],
$config['location'],
$queue,
(string) request()->headers->get('X-Cloudtasks-Taskname')
(string)$headerTaskName
);

CloudTasksApi::deleteTask($taskName);
Expand Down
45 changes: 17 additions & 28 deletions src/Config.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,21 +9,6 @@

class Config
{
public static function validate(array $config): void
{
if (empty($config['project'])) {
throw new Error(Errors::invalidProject());
}

if (empty($config['location'])) {
throw new Error(Errors::invalidLocation());
}

if (empty($config['service_account_email'])) {
throw new Error(Errors::invalidServiceAccountEmail());
}
}

/**
* @param Closure|string $handler
*/
Expand All @@ -42,25 +27,29 @@ public static function getHandler($handler): string
// (still) set to localhost. That will never work because Cloud Tasks
// should always call a public address / hostname to process tasks.
if (in_array($parse['host'], ['localhost', '127.0.0.1', '::1'])) {
throw new Exception(sprintf(
'Unable to push task to Cloud Tasks because the handler URL is set to a local host: %s. ' .
'This does not work because Google is not able to call the given local URL. ' .
'If you are developing on locally, consider using Ngrok or Expose for Laravel to expose your local ' .
'application to the internet.',
$handler
));
throw new Exception(
sprintf(
'Unable to push task to Cloud Tasks because the handler URL is set to a local host: %s. ' .
'This does not work because Google is not able to call the given local URL. ' .
'If you are developing on locally, consider using Ngrok or Expose for Laravel to expose your local ' .
'application to the internet.',
$handler
)
);
}

// When the application is running behind a proxy and the TrustedProxy middleware has not been set up yet,
// an error like [HttpRequest.url must start with 'https'] could be thrown. Since the handler URL must
// always be https, we will provide a little extra information on how to fix this.
if ($parse['scheme'] !== 'https') {
throw new Exception(sprintf(
'Unable to push task to Cloud Tasks because the hander URL is not https. Google Cloud Tasks ' .
'will only call safe (https) URLs. If you are running Laravel behind a proxy (e.g. Ngrok, Expose), make sure it is ' .
'as a trusted proxy. To quickly fix this, add the following to the [app/Http/Middleware/TrustProxies] middleware: ' .
'protected $proxies = \'*\';'
));
throw new Exception(
sprintf(
'Unable to push task to Cloud Tasks because the hander URL is not https. Google Cloud Tasks ' .
'will only call safe (https) URLs. If you are running Laravel behind a proxy (e.g. Ngrok, Expose), make sure it is ' .
'as a trusted proxy. To quickly fix this, add the following to the [app/Http/Middleware/TrustProxies] middleware: ' .
'protected $proxies = \'*\';'
)
);
}

$trimmedHandlerUrl = rtrim($handler, '/');
Expand Down
4 changes: 2 additions & 2 deletions src/DashboardService.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ public static function make(): DashboardService

private function getTaskBody(Task $task): string
{
$httpRequest = $task->getHttpRequest();
$httpRequest = $task->getHttpRequest() ?: $task->getAppEngineHttpRequest();

if (! $httpRequest instanceof HttpRequest) {
if (! $httpRequest) {
throw new Exception('Task does not have a HTTP request.');
}

Expand Down
5 changes: 5 additions & 0 deletions src/Errors.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,9 @@ public static function invalidServiceAccountEmail(): string
{
return 'Google Service Account email address not provided. This is needed to secure the handler so it is only accessible by Google. To fix this, set the STACKKIT_CLOUD_TASKS_SERVICE_EMAIL environment variable';
}

public static function serviceAccountOrAppEngine(): string
{
return 'A Google Service Account email or App Engine Request must be set. Set STACKKIT_CLOUD_TASKS_SERVICE_EMAIL or STACKKIT_APP_ENGINE_TASK';
}
}
Loading

0 comments on commit e5a6cb3

Please sign in to comment.