Skip to content

Commit

Permalink
🚀 proof of concept
Browse files Browse the repository at this point in the history
  • Loading branch information
bnomei committed Jun 30, 2024
1 parent 2d80baa commit 734479f
Show file tree
Hide file tree
Showing 33 changed files with 1,481 additions and 246 deletions.
127 changes: 103 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
[![Maintainability](https://flat.badgen.net/codeclimate/maintainability/bnomei/kirby-mongodb)](https://codeclimate.com/github/bnomei/kirby-mongodb)
[![Twitter](https://flat.badgen.net/badge/twitter/bnomei?color=66d9ef)](https://twitter.com/bnomei)

Khulan is a cache driver and content cache for Kirby using MongoDB.
Khulan is a cache driver and content cache with NoSQL interface for Kirby using MongoDB.

## Commercial Usage

Expand All @@ -29,12 +29,27 @@ Khulan is a cache driver and content cache for Kirby using MongoDB.
- `git submodule add https://github.com/bnomei/kirby-mongodb.git site/plugins/kirby-mongodb` or
- `composer require bnomei/kirby-mongodb`

## MongoDB

There are various ways to install [MongoDB](https://www.mongodb.com/). This is one way to do it on localhost for Mac OSX
using Homebrew and MongoDB Atlas.

```sh
brew install mongodb-atlas
atlas setup # create account and sign-in
atlas deployments setup # -> select localhost
alias deployments start # start the local mongodb
```

## Usecase

The plugin caches all content files and keeps the cache up to date when you add/remove or update content. This cache
will be used when constructing page/file/user objects making everything that involves model faster (even the
Panel).

It will also allow you to query the content cache directly as a NoSQL database which might be very useful for some
use-cases like filtering or searching content.

## Setup

For each template you want to be cached you need to use a model to add the content cache logic using a trait.
Expand All @@ -44,30 +59,105 @@ For each template you want to be cached you need to use a model to add the conte
```php
class DefaultPage extends \Kirby\Cms\Page
{
use \Bnomei\Khulan;
use \Bnomei\ModelWithKhulan;
}
```

> Note: You can also use the trait for user models. File models are patched automatically.
## Cache Driver
## Kirby's Content goes NoSQL

```php
/** @var \Kirby\Cache\Cache $cache */
$cache = \Bnomei\Mongodb::singleton();
The plugin writes the content cache to a collection named `khulan` in the database. You can query this collection
directly. It is **not** wrapped in an Cache object. This allows you to treat all your Kirby content as a NoSQL database.

$cache->set('mykey', 'myvalue', 5); // ttl in minutes
$value = $cache->get('mykey');
```php
// using the collection
$document = khulan()->find(['uuid' => 'XXX']);

// find a single page by uuid string
$page = khulan((string) $myIdOrUUID);

// find a single page by uuid by field-key
$page = khulan(['uuid' => 'page://betterharder']);

// find all pages with a template
$pages = khulan(['template' => 'post']);

// find all pages that have another page linked
$pages = khulan([
'related[]' => ['$in' => ['page://fasterstronger']],
]);

// find all products in the category 'books' or 'movies'
// that had been modified within the last 7 days
$pages = khulan([
'template' => 'product',
'category[,]' => ['$in' => ['books', 'movies']],
'modified' => ['$gt' => time() - (7 * 24 * 60 * 60)]
]);
```

## MongoDB Client

You can access the underlying MongoDB client directly.

```php
/** @var \MongoDB\Client $client */
$client = \Bnomei\Mongodb::singleton()->client();
$client = mongodb();
$client = mongo()->client();

$collection = $client->selectCollection('my-database', 'my-collection');
```

## Cache

You can either use the *cache* directly or use it as a *cache driver* for Kirby.

The MongoDB based cache will, compared to the default
file-based cache, perform **worse**! This is to be expected as web-servers are optimized for handling requests to a
couple of hundred files and keep them in memory.

```php
$cache = mongo()->cache();

$client->listDatabases();
$cache->set('mykey', 'myvalue', 5); // ttl in minutes
$value = $cache->get('mykey');

$cache->set('tagging', [
'tags' => ['tag1', 'tag2'],
'value' => 'myvalue',
]);
```

As with regular Kirby cache you can also use the `getOrSet` method to wrap your time expensive code into a closure and
only execute it when the cache is empty.

```php
$cache = mongo()->cache();

$value = $cache->getOrSet('mykey', function() {
sleep(5); // like a API call, database query or filtering content
return 'myvalue';
}, 5);
```

Using the MongoDB-based cache will allow you to perform NoSQL queries on the cache and do advanced stuff like
filtering my tags or invalidating many cache entries at once.

```php
// NOTE: we are using the cacheCollection() method here
$collection = mongo()->cacheCollection();

// find all that have the tag 'tag1'
$documents = $collection->find([
'tags' => ['$in' => ['tag1']],
]);

// delete any cache entry older than 5 minutes
$deleteResult = $collection->deleteMany([
'expires_at' => ['$lt' => time() - 5*60]
]);
$deletedCount = $deleteResult->getDeletedCount();
```

## Using the Cache Driver in Kirby
Expand All @@ -88,26 +178,15 @@ return [
];
```

## Kirby's Content goes NoSQL

The plugin writes the content cache to a collection named `khulan` in the database. You can query this collection
directly. It is **not** wrapped in an Cache object. This allows you to treat all your Kirby content as a NoSQL database.

```php
$collection = \Bnomei\Mongodb::singleton()->collection();
// TODO get khulan collection
$khulan = khulan();
$whatGetReturned = $khulan->find(['uuid' => 'XXX']);
// TODO: more examples
$whatGetReturned = $khulan->find(['category' => 'books']);
```

## Settings

| bnomei.mongodb. | Default | Description |
|--------------------------|-------------|------------------------------------------------------------------------------|
| host | `127.0.0.1` | |
| port | `27017` | |
| username | `null` | |
| password | `null` | |
| database | `kirby` | |
| khulan.read | `true` | read from cache |
| khulan.write | `true` | write to cache |
| khulan.patch-files-class | `true` | monkey-patch the \Kirby\CMS\Files class to use Khulan for caching it content |
Expand Down
170 changes: 71 additions & 99 deletions classes/Khulan.php
Original file line number Diff line number Diff line change
@@ -1,130 +1,102 @@
<?php

declare(strict_types=1);

namespace Bnomei;

use Kirby\Filesystem\F;
use Kirby\Toolkit\Str;

trait Khulan
use Kirby\Cms\Collection;
use Kirby\Cms\File;
use Kirby\Cms\Files;
use Kirby\Cms\Page;
use Kirby\Cms\Pages;
use Kirby\Cms\Site;
use Kirby\Cms\User;
use Kirby\Cms\Users;
use Kirby\Toolkit\A;

class Khulan
{
/** @var bool */
private bool $khulanCacheWillBeDeleted;

public function hasKhulan(): bool
{
return true;
}

public function setBoostWillBeDeleted(bool $value): void
public static function index(): array
{
$this->khulanCacheWillBeDeleted = $value;
}

public function keyKhulan(?string $languageCode = null): string
{
$key = hash('xxh3', $this->id()); // can not use UUID since content not loaded yet
if (! $languageCode) {
$languageCode = kirby()->languages()->count() ? kirby()->language()->code() : null;
}
if ($languageCode) {
$key = $key.'-'.$languageCode;
$count = 0;
$hash = [];

// reading a field like the title will make sure
// that the page is loaded and cached
foreach (site()->index(true) as $page) {
$hash[] = $page->title()->value();
$count++;
}
// TODO: files, users

return $key;
return [
'count' => $count,
'hash' => hash('xxh3', implode('|', $hash)),
];
}

public function readContentCache(?string $languageCode = null): ?array
public static function flush(): bool
{
// TODO: change to direct client findByID
return Mongodb::singleton()->get(
$this->keyKhulan($languageCode).'-content',
null
);
}

public function readContent(?string $languageCode = null): array
{
// read from boostedCache if exists
$data = option('bnomei.mongodb.khulan.read') === false || option('debug') ? null : $this->readContentCache($languageCode);

// read from file and update boostedCache
if (! $data) {
$data = parent::readContent($languageCode);
if ($data && $this->khulanCacheWillBeDeleted !== true) {
$this->writeKhulan($data, $languageCode);
}
}
mongo()->contentCollection()->drop();

return $data;
return true;
}

public function writeKhulan(?array $data = null, ?string $languageCode = null): bool
public static function documentsToModels(iterable $documents): Collection|Pages|Files|Users|null
{
$cache = Mongodb::singleton();
if (! $cache || option('bnomei.mongodb.khulan.write') === false) {
return true;
}

$modified = $this->modified();
$documents = iterator_to_array($documents);

// in rare case file does not exists or is not readable
if ($modified === false) {
$this->deleteKhulan(); // whatever was in the cache is no longer valid

return false; // try again another time
if (empty($documents)) {
return null;
}

// TODO: change to direct client insertOne
return $cache->set(
$this->keyKhulan($languageCode).'-content',
array_filter($data, fn ($content) => $content !== null),
option('bnomei.mongodb.expire')
);
}
$models = [];

public function writeContent(array $data, ?string $languageCode = null): bool
{
// write to file and cache
return parent::writeContent($data, $languageCode) &&
$this->writeKhulan($data, $languageCode);
}

public function deleteKhulan(): bool
{
$cache = Mongodb::singleton();
if (! $cache) {
return true;
foreach ($documents as $document) {
$models[] = self::documentToModel($document);
}

$this->setBoostWillBeDeleted(true);

foreach (kirby()->languages() as $language) {
// TODO: change to direct client deleteByID
$cache->remove(
$this->keyKhulan($language->code()).'-content'
);
$models = array_filter($models, function ($obj) {
return $obj !== null;
});

$modelTypes = array_count_values(array_map(function ($document) {
return $document['modelType'];
}, $documents));

if (count($modelTypes) === 1) {
$modelType = array_key_first($modelTypes);
if ($modelType === 'file') {
return new Files($models);
} elseif ($modelType === 'user') {
return new Users($models);
} elseif ($modelType === 'page') {
return new Pages($models);
}
} else {
return new Collection($models);
}

// TODO: change to direct client deleteByID
$cache->remove(
$this->keyKhulan().'-content'
);

return true;
return null;
}

public function delete(bool $force = false): bool
public static function documentToModel($document = null): Page|File|User|Site|null
{
$cache = Mongodb::singleton();
if (! $cache) {
return parent::delete($force);
if (! $document) {
return null;
}

$success = parent::delete($force);
$this->deleteKhulan();
if ($document['modelType'] === 'file') {
return kirby()->file($document['id']);
} elseif ($document['modelType'] === 'user') {
return kirby()->user($document['id']);
} elseif ($document['modelType'] === 'site') {
return kirby()->site();
} elseif ($document['modelType'] === 'page') {
$document = iterator_to_array($document);
$id = A::get($document, 'uuid', A::get($document, 'id'));

return kirby()->page($id);
}

return $success;
return null;
}
}
2 changes: 1 addition & 1 deletion classes/KhulanFile.php
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,5 @@

class KhulanFile extends \Kirby\Cms\File
{
use Khulan;
// use ModelWithKhulan; // TODO: breaks stuff
}
2 changes: 1 addition & 1 deletion classes/KhulanPage.php
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,5 @@

class KhulanPage extends \Kirby\Cms\Page
{
use Khulan;
use ModelWithKhulan;
}
Loading

0 comments on commit 734479f

Please sign in to comment.