diff --git a/Makefile b/Makefile index 237889c40..50df5e755 100644 --- a/Makefile +++ b/Makefile @@ -29,6 +29,7 @@ docs_api: crystal docs --output=docs/static/api/dev cp -R docs/static/api/dev/ docs/static/api/0.1 cp -R docs/static/api/dev/ docs/static/api/0.2 + cp -R docs/static/api/dev/ docs/static/api/0.3 .PHONY: docs_site docs_site: diff --git a/docs/docs/the-marten-project/release-notes.md b/docs/docs/the-marten-project/release-notes.md index d18c54894..39cbd392a 100644 --- a/docs/docs/the-marten-project/release-notes.md +++ b/docs/docs/the-marten-project/release-notes.md @@ -8,7 +8,7 @@ Here are listed the release notes for each version of the Marten web framework. ## Marten 0.3 -* [Marten 0.3 release notes](./release-notes/0.3) _(under development)_ +* [Marten 0.3 release notes](./release-notes/0.3) ## Marten 0.2 diff --git a/docs/docs/the-marten-project/release-notes/0.3.md b/docs/docs/the-marten-project/release-notes/0.3.md index 98a6954cf..ed7060ef7 100644 --- a/docs/docs/the-marten-project/release-notes/0.3.md +++ b/docs/docs/the-marten-project/release-notes/0.3.md @@ -4,7 +4,7 @@ pagination_prev: null pagination_next: null --- -_Under development._ +_June 19, 2023._ ## Requirements and compatibility diff --git a/docs/docusaurus.config.js b/docs/docusaurus.config.js index bf85085d8..1ab290a59 100644 --- a/docs/docusaurus.config.js +++ b/docs/docusaurus.config.js @@ -52,7 +52,7 @@ const darkCodeTheme = require('prism-react-renderer/themes/dracula'); dropdownActiveClassDisabled: true, }, { - href: 'https://martenframework.com/docs/api/0.2/index.html', + href: 'https://martenframework.com/docs/api/0.3/index.html', label: 'API', position: 'right', }, diff --git a/docs/versioned_docs/version-0.3/assets.mdx b/docs/versioned_docs/version-0.3/assets.mdx new file mode 100644 index 000000000..f0eabd530 --- /dev/null +++ b/docs/versioned_docs/version-0.3/assets.mdx @@ -0,0 +1,15 @@ +--- +title: Assets +--- + +import DocCard from '@theme/DocCard'; + +Marten provides a set of helpers in order to help you manage and resolve assets (also known as "static files") such as images, Javascript files, CSS files, etc. + +## Guides + +
+
+ +
+
diff --git a/docs/versioned_docs/version-0.3/assets/introduction.md b/docs/versioned_docs/version-0.3/assets/introduction.md new file mode 100644 index 000000000..494bc7027 --- /dev/null +++ b/docs/versioned_docs/version-0.3/assets/introduction.md @@ -0,0 +1,178 @@ +--- +title: Assets handling +description: Learn how to handle assets. +sidebar_label: Introduction +--- + +Web applications generally need to serve "static files" or "assets": static images, Javascript files, CSS files, etc. Marten provides a set of helpers in order to help you manage assets, refer to them, and upload them to specific storages. + +## Idea and scope + +Asset files can be defined in two places: + +* they can be provided by [apps](../development/applications): for example, some apps need to rely on specific assets to provide full-featured UIs +* they can be defined in [specifically configured folders](../development/reference/settings#dirs) in projects + +This allows applications to be relatively independent and to rely on their own assets if they need to, while also allowing projects to define assets as part of their structure. + +When a project is deployed, it is expected that all these asset files will be "collected" to be placed to the final destination from which they will be served: this operation is made available through the use of the [`collectassets`](../development/reference/management-commands#collectassets) management command. This "destination" depends on your deployment strategy: it can be as simple as moving all these assets to a dedicated folder in your server (so that they can be served by your web server), or it can involve uploading these assets to an S3 or GCS bucket for example. + +:::info +The assets flow provided by Marten is **intentionally simple**. Indeed, Marten being a backend-oriented framework, can't account for all the ways assets can be packaged and/or bundled together. Some projects might require a webpack strategy to bundle assets, some might require a fingerprinting step on top of that, and others might need something entirely different. How these toolchains are configured or set up is left to the discretion of web application developers; it is just expected that these operations will be applied _before_ the [`collectassets`](../development/reference/management-commands#collectassets) management command is executed. +::: + +Once assets have been "collected", it is possible to generate their URLs through the use of dedicated helpers: + +* by using the [assets engine](pathname:///api/0.3/Marten/Asset/Engine.html#url(filepath%3AString)%3AString-instance-method) in Crystal +* by using the [`asset`](../templates/reference/tags#asset) tag in templates + +The way these asset URLs are generated depends on the configured [asset storage](../development/reference/settings#storage). + +## Configuring assets + +Assets can be configured through the use of the [assets settings](../development/reference/settings#assets-settings), which are available under the `assets` namespace. + +An example assets configuration might look like this: + +```crystal +config.assets.root = "assets" +config.assets.url = "/assets/" +``` + +### Assets storage + +One of the most important asset settings is the [`storage`](../development/reference/settings#storage) one. Indeed, Marten uses a file storage mechanism to perform file operations related to assets (like uploading files, generating URLs, etc) by leveraging a standardized API. By default, assets use the [`Marten::Core::Store::FileSystem`](pathname:///api/0.3/Marten/Core/Storage/FileSystem.html) storage backend, which ensures that assets files are collected and placed to a specific folder in the local file system: this allows these files to then be served by a web server such as Nginx for example. + +### Assets root directory + +This directory - which can be configured through the use of the [`root`](../development/reference/settings#root) setting - corresponds to the absolute path where collected assets will be persisted (when running the [`collectassets`](../development/reference/management-commands#collectassets) command). By default, assets will be persisted in a folder that is relative to the Marten project's directory. Obviously, this folder should be empty before running the `collectassets` command in order to not overwrite existing files. The default value is `assets`. + +### Assets URL + +The asset URL is used when generating URLs for assets. This base URL will be used by the default [`Marten::Core::Store::FileSystem`](pathname:///api/0.3/Marten/Core/Storage/FileSystem.html) storage to construct asset URLs. For example, requesting a `css/App.css` asset might generate a `/assets/css/App.css` URL. The default value is `/assets/`. + +### Asset directories + +By default, Marten will collect asset files that are defined under an `assets` folder in [application](../development/applications) directories. That being said, your project will probably have asset files that are not associated with a particular app. That's why you can also define an array of additional directories where assets should be looked for. + +This array of directories can be defined through the use of the [`dirs`](../development/reference/settings#dirs) assets setting: + +```crystal +config.assets.dirs = [ + Path["src/path1/assets"], + :"src/path2/assets", +] +``` + +## Resolving asset URLs + +As mentioned previously, assets are collected and persisted in a specific storage. When building HTML [templates](../templates/introduction), you will usually need to "resolve" the URL of assets to generate the absolute URLs that should be inserted into stylesheet or script tags (for example). + +One possible way to do so is to leverage the [`asset`](../templates/reference/tags#asset) template tag. This template tag takes a single argument corresponding to the relative path of the asset you want to resolve, and it outputs the absolute URL of the asset (depending on your assets configuration). + +For example: + +```html + +``` + +In the above snippet, the `app/app.css` asset could be resolved to `/assets/app/app.css` (depending on the configuration of the project obviously). + +It is also possible to resolve asset URLs programmatically in Crystal. To do so, you can leverage the [`#url`](pathname:///api/0.3/Marten/Asset/Engine.html#url(filepath%3AString)%3AString-instance-method) method of the Marten assets engine: + +```crystal +Marten.assets.url("app/app.css") # => "/assets/app/app.css" +``` + +## Serving assets in development + +Marten provides a handler that you can use to serve assets in development environments only. This handler ([`Marten::Handlers::Defaults::Development::ServeAsset`](pathname:///api/0.3/Marten/Handlers/Defaults/Development/ServeAsset.html)) is automatically mapped to a route when creating new projects through the use of the [`new`](../development/reference/management-commands#new) management command: + +```crystal +Marten.routes.draw do + # Other routes... + + if Marten.env.development? + path "#{Marten.settings.assets.url}", Marten::Handlers::Defaults::Development::ServeAsset, name: "asset" + end +end +``` + +As you can see, this route will automatically use the URL that is configured as part of the [`url`](../development/reference/settings#url) asset setting. For example, this means that an `app/app.css` asset would be served by the `/assets/app/app.css` route in development if the [`url`](../development/reference/settings#url) setting is set to `/assets/`. + +:::warning +It is very important to understand that this handler should **only** be used in development environments. Indeed, the [`Marten::Handlers::Defaults::Development::ServeAsset`](pathname:///api/0.3/Marten/Handlers/Defaults/Development/ServeAsset.html) handler does not require assets to have been collected beforehand through the use of the [`collectassets`](../development/reference/management-commands#collectassets) management command. This means that it will try to find assets in your applications' `assets` directories and in the directories configured in the [`dirs`](../development/reference/settings#dirs) setting. This mechanism is helpful in development, but it is not suitable for production environments since it is inneficient and (probably) insecure. +::: + +## Serving assets in production + +At deployment time, you will need to run the [`collectassets`](../development/reference/management-commands#collectassets) management command to collect all the available assets from the applications' `assets` directories and from the directories configured in the [`dirs`](../development/reference/settings#dirs) setting. This command will identify and "collect" those assets, and ensure they are "uploaded" into their final destination based on the storage that is currently used. + +:::tip +The [`collectassets`](../development/reference/management-commands#collectassets) management command should be executed _after_ your assets have been bundled and packaged. For example, your project could use a [gulp](https://gulpjs.com/) pipeline to compile your assets, minify them, and place them into a `src/app/assets/build` directory. Assuming that this directory is also specified in the [`dirs`](../development/reference/settings#dirs) setting, these prepared assets would also be collected and uploaded into the configured storage. Which would allow you to then refer to them from your project's templates. + +Obviously, every project is different and might use different tools and a different deployment pipeline, but the overall strategy would remain the same. +::: + +It should be noted that there are many ways to serve assets in production. Again, every deployment situation will be different, but we can identify a few generic strategies. + +### Serving assets from a web server + +As mentioned previously, Marten uses a file storage mechanism to perform file operations related to assets and to "collect" them. By default, assets use the [`Marten::Core::Store::FileSystem`](pathname:///api/0.3/Marten/Core/Storage/FileSystem.html) storage backend, which ensures that assets files are collected and placed into a specific folder in the local file system. This allows these assets to easily be served by a local web server if you have one properly configured. + +For example, you could use a web server like [Apache](https://httpd.apache.org/) or [Nginx](https://nginx.org) to serve your collected assets. The way to configure these web servers will obviously vary from one solution to another, but you will likely need to define a location whose URL matches the [`url`](../development/reference/settings#url) setting value and that serves files from the folder where assets were collected (the [`root`](../development/reference/settings#root) folder). + +For example, a [Nginx](https://nginx.org) server configuration allowing to serve assets under a `/assets` location could look like this: + +```conf +server { + listen 443 ssl; + server_name myapp.example.com; + + gzip on; + gzip_disable "msie6"; + gzip_vary on; + gzip_proxied any; + gzip_comp_level 6; + gzip_buffers 16 8k; + gzip_http_version 1.1; + gzip_min_length 256; + gzip_types text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript application/vnd.ms-fontobject application/x-font-ttf font/opentype image/svg+xml image/x-icon; + + error_log /var/log/nginx/myapp_error.log; + access_log /var/log/nginx/myapp_access.log; + + location /assets/ { + expires 365d; + alias /myapp/assets/; + } +} +``` + +### Serving assets from a cloud service or CDN + +To serve assets from a cloud storage (like Amazon's S3 or GCS) and (optionally) a CDN (Content Delivery Network), you will likely need to write a custom file storage and set the [`storage`](../development/reference/settings#storage) setting accordingly. The advantage of doing so is that you are basically delegating the responsibility of serving assets to a dedicated cloud storage, which can often translate into faster-loading pages for your end users. + +:::info +Marten does not provide file storage implementations for the most frequently encountered cloud storage solutions presently. This is something that is planned for future releases though. +::: + +Writing a custom file storage implementation will involve subclassing the [`Marten::Core::Storage::Base`](pathname:///api/0.3/Marten/Core/Storage/Base.html) abstract class and implementing a set of mandatory methods. The main difference compared to a "local file system" storage here is that you would need to make use of the API of the chosen cloud storage to perform low-level file operations (such as reading a file's content, verifying that a file exists, or generating a file URL). + +### Serving assets using a middleware + +There are some situations where it is not possible to easily configure a web server such as [Nginx](https://nginx.org) or a third-party service (like Amazon's S3 or GCS) to serve your assets directly. To palliate this, Marten provides the [`Marten::Middleware::AssetServing`](../handlers-and-http/reference/middlewares#asset-serving-middleware) middleware. + +The purpose of this middleware is to distribute collected assets stored under the configured assets root ([`assets.root`](../development/reference/settings#root) setting). These assets are assumed to have been collected using the [`collectassets`](../../development/reference/management-commands#collectassets) management command, and it is also assumed that a "local file system" storage (such as [`Marten::Core::Store::FileSystem`](pathname:///api/0.3/Marten/Core/Storage/FileSystem.html)) is used. + +In order to use this middleware, you can "insert" the corresponding class at the beginning of the [`middleware`](../development/reference/settings#middleware) setting when defining production settings. For example: + +```crystal +Marten.configure :production do |config| + config.middleware.unshift(Marten::Middleware::AssetServing) + + # Other settings... +end +``` + +It is important to note that the [`assets.url`](../development/reference/settings#url) setting must align with the Marten application domain or correspond to a relative URL path (e.g., /assets/) for this middleware to work correctly. This guarantees proper mapping and accessibility of the assets within the application, allowing them to be served by this middleware. diff --git a/docs/versioned_docs/version-0.3/authentication.mdx b/docs/versioned_docs/version-0.3/authentication.mdx new file mode 100644 index 000000000..882cc8731 --- /dev/null +++ b/docs/versioned_docs/version-0.3/authentication.mdx @@ -0,0 +1,23 @@ +--- +title: Authentication +--- + +import DocCard from '@theme/DocCard'; + +Marten provides the ability to generate projects with a built-in authentication system that handles basic user management needs. This section gives detail on how this authentication system works out of the box, and how you can extend it to suit your project's needs. + +## Guides + +
+
+ +
+
+ +## Reference + +
+
+ +
+
diff --git a/docs/versioned_docs/version-0.3/authentication/introduction.md b/docs/versioned_docs/version-0.3/authentication/introduction.md new file mode 100644 index 000000000..3031d9f76 --- /dev/null +++ b/docs/versioned_docs/version-0.3/authentication/introduction.md @@ -0,0 +1,207 @@ +--- +title: Introduction to authentication +description: Learn how to set up authentication for your Marten project. +sidebar_label: Introduction +--- + +Marten allows the generation of new projects with a built-in authentication application that handles basic user management needs. You can then extend and adapt this application so that it accommodates your project's needs. + +## Overview + +Marten's [`new`](../development/reference/management-commands#new) management allows the generation of projects with a built-in `auth` application. This application is part of the created project: it provides the necessary [models](../models-and-databases), [handlers](../handlers-and-http), [schemas](../schemas), [emails](../emailing), and [templates](../templates) allowing to authenticate users with email addresses and passwords, while also supporting standard password reset flows. On top of that, an `Auth::User` model is automatically generated for your newly created projects. Since this model is also part of your project, this means that it's possible to easily add new fields to it and generate migrations for it as well. + +Here is the list of responsibilities of the generated authentication application: + +* Signing up users +* Signing in users +* Signing out users +* Allowing users to reset their passwords +* Allowing users to access a basic profile page + +Internally, this authentication application relies on the official [`marten-auth`](https://github.com/martenframework/marten-auth) shard. This shard implements low-level authentication operations (such as authenticating user credentials, generating securely encrypted passwords, generating password reset tokens, etc). + +## Generating projects with authentication + +Generating new projects with authentication can be easily achieved by leveraging the `--with-auth` option of the [`new`](../development/reference/management-commands#new) management command. + +For example: + +```bash +marten new project myblog --with-auth +``` + +When using this option, Marten will generate an `auth` [application](../development/applications) under the `src/auth` folder of your project. As mentioned previously, this application provides a set of [models](../models-and-databases), [handlers](../handlers-and-http), [schemas](../schemas), [emails](../emailing), and [templates](../templates) that implement basic authentication operations. + +You can test the generated authentication application by going to your application at [http://localhost:8000/auth/signup](http://localhost:8000/auth/signup) after having started the Marten development server (using `marten serve`). + +:::info +You can see the full list of files generated for the `auth` application in [Generated files](./reference/generated-files). +::: + +## Usage + +This section covers the basics of how to use the `auth` application - powered by [`marten-auth`](https://github.com/martenframework/marten-auth) - that is generated when creating projects with the `--with-auth` option. + +### The `User` model + +The `auth` application defines a single `Auth::User` model that inherits its fields from the abstract `MartenAuth::User` model. As such, this model automatically provides the following fields: + +* `id` - a [`big_int`](../models-and-databases/reference/fields#big_int) field containing the primary key of the user +* `email` - an [`email`](../models-and-databases/reference/fields#email) field containing the user's email address +* `password` - a [`string`](../models-and-databases/reference/fields#string) field containing the user's encrypted password +* `created_at` - a [`date_time`](../models-and-databases/reference/fields#date_time) field containing the user creation date +* `updated_at` - a [`date_time`](../models-and-databases/reference/fields#date_time) field containing the last user modification date + +### Retrieving the current user + +Projects that are generated with the `auth` application automatically make use of a middleware (`MartenAuth::Middleware`) that ensures that the currently authenticated user ID is associated with the current request. This means that given a specific HTTP request (instance of [`Marten::HTTP::Request`](pathname:///api/0.3/Marten/HTTP/Request.html)), it is possible to identify which user is signed-in or not. Concretely, the following methods are made available on the standard [`Marten::HTTP::Request`](pathname:///api/0.3/Marten/HTTP/Request.html) object in order to interact with the currently signed-in user: + +| Method | Description | +| --- | --- | +| `#user_id` | Returns the current user ID associated with the considered request, or `nil` if there is no authenticated user. | +| `#user` | Returns the user associated with the request, or `nil` if there is no authenticated user. | +| `#user!` | Returns the user associated with the request, or raise `NilAssertionError` if there is no authenticated user. | +| `#user?` | Returns `true` if a user is authenticated for the request. | + +This makes it possible to easily check whether a user is authenticated in handlers in order to implement different logic. For example: + +```crystal +class MyHandler < Marten::Handler + def get + if request.user? + respond "User ##{request.user!.id} is signed-in" + else + respond "No signed-in user" + end + end +end +``` + +### Creating users + +Creating a user is as simple as initializing an instance of the `Auth::User` model and defining its properties. That being said it is important to note that the user's password (`password` field) must be set using the `#set_password` method: this will ensure that the _raw_ password you provide to this method is properly encrypted and that the resulting hash is assigned to the `password` field. Because of this, you should not attempt to manipulate the `password` field attribute of user records directly. + +For example: + +```crystal +user = Auth::User.new(email: "test@example.com") do |user| + user.set_password("insecure") +end + +user.save! +``` + +### Authenticating users + +Authentication is the act of verifying a user's credentials. This capability is provided by the [`marten-auth`](https://github.com/martenframework/marten-auth) shard through the use of the `MartenAuth#authenticate` method: this method tries to authenticate the user associated identified by a natural key (typically, an email address) and check that the given raw password is valid. The method returns the corresponding user record if the authentication is successful. Otherwise, it returns `nil` if the credentials can't be verified because the user does not exist or because the password is invalid. + +For example: + +```crystal +user = MartenAuth.authenticate("test@example.com", "insecure") +if user + puts "User credentials are valid!" +else + puts "User credentials are not valid!" +end +``` + +:::caution +It is important to realize that this method _only_ verifies user credentials. **It does not sign in users** for a specific request. Signing in users (and attaching them to the current session) is handled by the `#sign_in` method, which is discussed in [Signing in users](#signing-in-users). +::: + +:::info +The `MartenAuth#authenticate` method is automatically used by the handlers that are generated for your `auth` application before signing in users. +::: + +### Signing in users + +Signing in a user is the act of attaching it to the current session - after having verified that the associated credentials are valid (see [Authenticating users](#authenticating-users)). This capability is provided by the [`marten-auth`](https://github.com/martenframework/marten-auth) shard through the use of the `MartenAuth#sign_in` method: This method takes a request object (instance of [`Marten::HTTP::Request`](pathname:///api/0.3/Marten/HTTP/Request.html)) and a user record as arguments and ensures that the user ID is attached to the current session so that they do not have to reauthenticate for every request. + +For example: + +```crystal +class MyHandler < Marten::Handler + def post + user = MartenAuth.authenticate(request.data["email"].to_s, request.data["password"].to_s) + + if user + MartenAuth.sign_in(request, user) + redirect reverse("auth:profile") + else + redirect reverse("auth:sign_in") + end + end +end +``` + +:::caution +It is important to understand that this method is intended to be used for a user record whose credentials were validated using the `#authenticate` method beforehand. See [Authenticating users](#authenticating-users) for more details. +::: + +### Signing out users + +The ability to sign out users is provided by the [`marten-auth`](https://github.com/martenframework/marten-auth) shard through the use of the `MartenAuth#sign_out` method: this method takes a request object (instance of [`Marten::HTTP::Request`](pathname:///api/0.3/Marten/HTTP/Request.html)) as argument, removes the authenticated user ID from the current request, and flushes the associated session data. + +For example: + +```crystal +class MyHandler < Marten::Handler + def get + MartenAuth.sign_out(request) + redirect reverse("auth:sign_in") + end +end +``` + +### Changing a user's password + +The ability to change a user password is provided by the `#set_password` method of the `Auth::User` model (which is inherited from the `MartenAuth::User` abstract class that is provided by the [`marten-auth`](https://github.com/martenframework/marten-auth) shard). + +For example: + +```crystal +use = User.get!(email: "test@example.com") +user.set_password("insecure") +user.save! +``` + +:::info +Passwords are encrypted using [`Crypto::Bcrypt`](https://crystal-lang.org/api/Crypto/Bcrypt.html). +::: + +As mentioned previously, you should not attempt to manipulate the `password` field directly: this field contains the hash value that results from the encryption of the raw password. + +### Limiting access to signed-in users + +Limiting access to signed-in users can easily be achieved by leveraging the `#user?` method that is available from [`Marten::HTTP::Request`](pathname:///api/0.3/Marten/HTTP/Request.html) objects. Using this method, you can easily implement [`#before_dispatch`](../handlers-and-http/introduction#before_dispatch) handler callbacks in order to redirect anonymous users to a sign-in page or to an error page. + +For example: + +```crystal +class UserProfileHandler < Marten::Handler + before_dispatch :require_signed_in_user + + def get + render "auth/profile.html" { user: request.user } + end + + private def require_signed_in_user + redirect reverse("auth:sign_in") unless request.user? + end +end +``` + +It should be noted that the `auth` application generated for your project already contains an `Auth::RequireSignedInUser` concern module that you can include in your handlers in order to ensure that they can only be accessed by signed-in users (and that anonymous users are redirected to the sign-in page). + +For example: + +```crystal +class UserProfileHandler < Marten::Handler + include Auth::RequireSignedInUser + + def get + render "auth/profile.html" { user: request.user } + end +end +``` diff --git a/docs/versioned_docs/version-0.3/authentication/reference/generated-files.md b/docs/versioned_docs/version-0.3/authentication/reference/generated-files.md new file mode 100644 index 000000000..adcb4d8ae --- /dev/null +++ b/docs/versioned_docs/version-0.3/authentication/reference/generated-files.md @@ -0,0 +1,57 @@ +--- +title: Generated files +description: Generated files reference. +--- + +This page provides a reference of the files that are generated for the `auth` application when running the [`new`](../../development/reference/management-commands#new) management command with the `--with-auth` option. + +## Application + +The `auth` application is generated under the `src/auth` folder. In addition to the abstractions mentioned below, this folder defines the following top-level files: + +* `app.cr` - The entrypoint of the `auth` application, where all the other abstractions are required +* `cli.cr` - The CLI entrypoint of the `auth` application, where CLI-related abstractions (like migrations) are required +* `routes.cr` - The `auth` application routes map + +### Emails + +* `password_reset_email.cr` - Defines the [email](../../emailing) that is sent as part of the user password reset flow + +### Handlers + +* `handlers/concerns/require_anonymous_user.cr` - A concern that ensures that a handler can only be accessed by anonymous users +* `handlers/concerns/require_signed_in_user.cr` - A concern that ensures that a handler can only be accessed by signed-in users +* `handlers/password_reset_confirm_handler.cr` - A hander that handles resetting a user's password as part of the password reset flow +* `handlers/password_reset_initiate_handler.cr` - A hander that initiates the password reset flow for a given user +* `handlers/profile_handler.cr` - A handler that displays the currently signed-in user profile +* `handlers/sign_in_handler.cr` - A handler that allows users to sign in +* `handlers/sign_out_handler.cr` - A handler that allows users to sign out +* `handlers/sign_up_handler.cr` - A handler that allows users to sign up + +### Migrations + +* `migrations/0001_create_auth_user_table.cr` - Allows to create the table of the `Auth::User` model + +### Models + +* `models/user.cr` - Defines the main `Auth::User` model + +### Schemas + +* `schemas/password_reset_confirm_schema.cr` - A schema that allows a user to reset their password +* `schemas/password_reset_initiate_schema.cr` - A schema that allows a user to initiate the password reset flow +* `schemas/sign_in_schema.cr` - A schema used to sign in users +* `schemas/sign_up_schema.cr` - A schema used to sign up users + +### Templates + +* `templates/auth/emails/password_reset.html` - The template of the password reset email +* `templates/auth/password_reset_confirm.html` - The template used to let users reset their passwords +* `templates/auth/password_reset_initiate.html` - The template used to let users initiate the password reset flow +* `templates/auth/profile.html` - The template of the user profile +* `templates/auth/sign_in.html` - The sign in page template +* `templates/auth/sign_up.html` - The sign up page template + +## Specs + +All the previously mentioned abstractions have associated specs that are defined under the `spec/auth` folder. diff --git a/docs/versioned_docs/version-0.3/caching.mdx b/docs/versioned_docs/version-0.3/caching.mdx new file mode 100644 index 000000000..d0e37df10 --- /dev/null +++ b/docs/versioned_docs/version-0.3/caching.mdx @@ -0,0 +1,31 @@ +--- +title: Caching +--- + +import DocCard from '@theme/DocCard'; + +Marten provides native support for caching, enabling you to store the outcome of resource-intensive operations and bypass performing them for each request. This encompasses basic caching abilities and advanced features like template fragment caching. + +## Guides + +
+
+ +
+
+ +## How-to's + +
+
+ +
+
+ +## Reference + +
+
+ +
+
diff --git a/docs/versioned_docs/version-0.3/caching/how-to/create-custom-cache-stores.md b/docs/versioned_docs/version-0.3/caching/how-to/create-custom-cache-stores.md new file mode 100644 index 000000000..d79810b6b --- /dev/null +++ b/docs/versioned_docs/version-0.3/caching/how-to/create-custom-cache-stores.md @@ -0,0 +1,147 @@ +--- +title: Create cache stores +description: How to create custom cache stores. +--- + +Marten lets you easily create custom [cache stores](../introduction#configuration-and-cache-stores) that you can then use as part of your application when it comes to perform caching operations. + +## Basic store definition + +Defining a cache store is as simple as creating a class that inherits from the [`Marten::Caching::Store::Base`](pathname:///api/0.3/Marten/Cache/Store/Base.html) abstract class and that implements the following methods: + +* [`#clear`](pathname:///api/0.3/Marten/Cache/Store/Base.html#clear-instance-method) - called when clearing the cache +* [`#decrement`](pathname:///api/0.3/Marten/Cache/Store/Base.html#decrement(key%3AString%2Camount%3AInt32%3D1%2Cexpires_at%3ATime|Nil%3Dnil%2Cexpires_in%3ATime%3A%3ASpan|Nil%3Dnil%2Cversion%3AInt32|Nil%3Dnil%2Crace_condition_ttl%3ATime%3A%3ASpan|Nil%3Dnil%2Ccompress%3ABool|Nil%3Dnil%2Ccompress_threshold%3AInt32|Nil%3Dnil)%3AInt-instance-method) - called when decrementing an integer value in the cache +* [`#delete_entry`](pathname:///http://localhost:3000/docs/api/0.3/Marten/Cache/Store/Base.html#delete_entry%28key%3AString%29%3ABool-instance-method) - called when deleting an entry from the cache +* [`#increment`](pathname:///api/0.3/Marten/Cache/Store/Base.html#increment(key%3AString%2Camount%3AInt32%3D1%2Cexpires_at%3ATime|Nil%3Dnil%2Cexpires_in%3ATime%3A%3ASpan|Nil%3Dnil%2Cversion%3AInt32|Nil%3Dnil%2Crace_condition_ttl%3ATime%3A%3ASpan|Nil%3Dnil%2Ccompress%3ABool|Nil%3Dnil%2Ccompress_threshold%3AInt32|Nil%3Dnil)%3AInt-instance-method) - called when incrementing an integer value in the cache +* [`#read_entry`](pathname:///api/0.3/Marten/Cache/Store/Base.html#read_entry(key%3AString)%3AString|Nil-instance-method) - called when reading an entry in the cache +* [`#write_entry`](pathname:///api/0.3/Marten/Cache/Store/Base.html#write_entry(key%3AString%2Cvalue%3AString%2Cexpires_in%3ATime%3A%3ASpan|Nil%3Dnil%2Crace_condition_ttl%3ATime%3A%3ASpan|Nil%3Dnil)-instance-method) - called when writing an entry to the cache + +For example, the following snippet implements an in-memory store that persists cache entries in a hash: + +```crystal +class MemoryStore < Marten::Cache::Store::Base + @data = {} of String => String + + def initialize( + @namespace : String? = nil, + @expires_in : Time::Span? = nil, + @version : Int32? = nil, + @compress = false, + @compress_threshold = DEFAULT_COMPRESS_THRESHOLD + ) + super + end + + def clear : Nil + @data.clear + end + + def decrement( + key : String, + amount : Int32 = 1, + expires_at : Time? = nil, + expires_in : Time::Span? = nil, + version : Int32? = nil, + race_condition_ttl : Time::Span? = nil, + compress : Bool? = nil, + compress_threshold : Int32? = nil + ) : Int + apply_increment( + key, + amount: -amount, + expires_at: expires_at, + expires_in: expires_in, + version: version, + race_condition_ttl: race_condition_ttl, + compress: compress, + compress_threshold: compress_threshold + ) + end + + def increment( + key : String, + amount : Int32 = 1, + expires_at : Time? = nil, + expires_in : Time::Span? = nil, + version : Int32? = nil, + race_condition_ttl : Time::Span? = nil, + compress : Bool? = nil, + compress_threshold : Int32? = nil + ) : Int + apply_increment( + key, + amount: amount, + expires_at: expires_at, + expires_in: expires_in, + version: version, + race_condition_ttl: race_condition_ttl, + compress: compress, + compress_threshold: compress_threshold + ) + end + + private getter data + + private def apply_increment( + key : String, + amount : Int32 = 1, + expires_at : Time? = nil, + expires_in : Time::Span? = nil, + version : Int32? = nil, + race_condition_ttl : Time::Span? = nil, + compress : Bool? = nil, + compress_threshold : Int32? = nil + ) + normalized_key = normalize_key(key.to_s) + entry = deserialize_entry(read_entry(normalized_key)) + + if entry.nil? || entry.expired? || entry.mismatched?(version || self.version) + write( + key: key, + value: amount.to_s, + expires_at: expires_at, + expires_in: expires_in, + version: version, + race_condition_ttl: race_condition_ttl, + compress: compress, + compress_threshold: compress_threshold + ) + amount + else + new_amount = entry.value.to_i + amount + entry = Entry.new(new_amount.to_s, expires_at: entry.expires_at, version: entry.version) + write_entry(normalized_key, serialize_entry(entry)) + new_amount + end + end + + private def delete_entry(key : String) : Bool + deleted_entry = @data.delete(key) + !!deleted_entry + end + + private def read_entry(key : String) : String? + data[key]? + end + + private def write_entry( + key : String, + value : String, + expires_in : Time::Span? = nil, + race_condition_ttl : Time::Span? = nil + ) + data[key] = value + true + end +end +``` + +## Enabling the use of custom cache stores + +Custom cache store can be used by assigning an instance of the corresponding class to the [`cache_store`](../../development/reference/settings#cache-store1) setting. + +For example: + +```crystal +config.cache_store = MemoryStore.new +``` diff --git a/docs/versioned_docs/version-0.3/caching/introduction.md b/docs/versioned_docs/version-0.3/caching/introduction.md new file mode 100644 index 000000000..dedc8cf0a --- /dev/null +++ b/docs/versioned_docs/version-0.3/caching/introduction.md @@ -0,0 +1,153 @@ +--- +title: Introduction to caching +description: Learn how to leverage caching in a Marten project. +sidebar_label: Introduction +--- + +Marten provides a set of features allowing you to leverage caching as part of your application. By using caching, you can save the result of expensive operations so that you don't have to perform them for every request. + +## Configuration and cache stores + +In order to be able to leverage caching in your application, you need to configure a "cache store". A cache store allows interacting with the underlying cache system and performing basic operations such as fetching cached entries, writing new entries, etc. Depending on the chosen cache store, these operations could be performed in-memory or by leveraging external caching systems such as [Redis](https://redis.io) or [Memcached](https://memcached.org). + +The global cache store used by Marten can be configured by leveraging the [`cache_store`](../development/reference/settings#cache_store) setting. All the available cache stores are listed in the [cache store reference](./reference/stores.md). + +For example, the following configuration configures an in-memory cache as the global cache: + +```crystal +Marten.configure do |config| + config.cache_store = Marten::Cache::Store::Memory.new.new(expires_in: 24.hours) +end +``` + +:::info +By default, Marten uses an in-memory cache (instance of [`Marten::Cache::Store::Memory`](pathname:///api/0.3/Marten/Cache/Store/Memory.html)). Note that this simple in-memory cache does not allow to perform cross-process caching since each process running your app will have its own private cache instance. In situations where you have multiple separate processes running your application, it's preferable to use a proper caching system such as [Redis](https://redis.io) or [Memcached](https://memcached.org), which can be done by leveraging respectively the [`marten-redis-cache`](https://github.com/martenframework/marten-redis-cache) or [`marten-memcached-cache`](https://github.com/martenframework/marten-memcached-cache) shards. + +In testing environments, you could configure your project so that it uses an instance of [`Marten::Cache::Store::Null`](pathname:///api/0.3/Marten/Cache/Store/Null.html) as the global cache. This approach can be helpful when caching is not necessary, but you still want to ensure that your code is passing through the caching interface. +::: + +## Low-level caching + +### Basic usage + +Low-level caching allows you to interact directly with the global cache store and perform caching operations. To do that, you can access the global cache store by calling the [`Marten#cache`](pathname:///api/0.3/Marten.html#cache%3ACache%3A%3AStore%3A%3ABase-class-method) method. + +The main way to put new values in cache is to leverage the [`#fetch`](pathname:///api/0.3/Marten/Cache/Store/Base.html#fetch(key%3AString|Symbol%2Cexpires_at%3ATime|Nil%3Dnil%2Cexpires_in%3ATime%3A%3ASpan|Nil%3Dnil%2Cversion%3AInt32|Nil%3Dnil%2Cforce%3Dfalse%2Crace_condition_ttl%3ATime%3A%3ASpan|Nil%3Dnil%2Ccompress%3ABool|Nil%3Dnil%2Ccompress_threshold%3AInt32|Nil%3Dnil%2C%26)%3AString|Nil-instance-method) method, which is provided on all cache stores. This method allows fetching data from the cache by using a specific key: if an entry exists for this key in cache, then the data is returned. Otherwise the return value of the block (that _must_ be specified when calling [`#fetch`](pathname:///api/0.3/Marten/Cache/Store/Base.html#fetch(key%3AString|Symbol%2Cexpires_at%3ATime|Nil%3Dnil%2Cexpires_in%3ATime%3A%3ASpan|Nil%3Dnil%2Cversion%3AInt32|Nil%3Dnil%2Cforce%3Dfalse%2Crace_condition_ttl%3ATime%3A%3ASpan|Nil%3Dnil%2Ccompress%3ABool|Nil%3Dnil%2Ccompress_threshold%3AInt32|Nil%3Dnil%2C%26)%3AString|Nil-instance-method)) is written to the cache and returned. This method supports a few additional arguments that allow to further customize how the entry is written to the cache (eg. the expiry time associated with the entry). + +For example: + +```crystal +Marten.cache.fetch("mykey", expires_in: 4.hours) do + "myvalue" +end +``` + +### Reading from and writing to the cache + +It is worth mentioning that you can also explicitly read from the cache and write to the cache by leveraging the [`#read`](pathname:///api/0.3/Marten/Cache/Store/Base.html#read(key%3AString|Symbol%2Cversion%3AInt32|Nil%3Dnil)%3AString|Nil-instance-method) and [`#write`](pathname:///api/0.3/Marten/Cache/Store/Base.html#write(key%3AString|Symbol%2Cvalue%3AString%2Cexpires_at%3ATime|Nil%3Dnil%2Cexpires_in%3ATime%3A%3ASpan|Nil%3Dnil%2Cversion%3AInt32|Nil%3Dnil%2Crace_condition_ttl%3ATime%3A%3ASpan|Nil%3Dnil%2Ccompress%3ABool|Nil%3Dnil%2Ccompress_threshold%3AInt32|Nil%3Dnil)-instance-method) methods respectively. Verifying that a key exists can be done using the [`#exists?`](pathname:///api/0.3/Marten/Cache/Store/Base.html#exists%3F(key%3AString|Symbol%2Cversion%3AInt32|Nil%3Dnil)%3ABool-instance-method) method. + +For example: + +```crystal +# No entry in the cache yet. +Marten.cache.read("foo") # => nil +Marten.cache.exists?("foo") => false + +# Let's add the entry to the cache. +Marten.cache.write("foo", "bar", expires_in: 10.minutes) => true + +# Let's read from the cache. +Marten.cache.read("foo") # => "bar" +Marten.cache.exists?("foo") => true +``` + +### Deleting an entry from the cache + +Deleting an entry from the cache is made possible through the use of the [`#delete`](pathname:///api/0.3/Marten/Cache/Store/Base.html#delete(key%3AString|Symbol)%3ABool-instance-method) method. This method takes the key of the entry to delete as argument and returns a boolean indicating whether an entry was actually deleted. + +For example: + +```crystal +# No entry in the cache yet. +Marten.cache.delete("foo") # => false + +# Let's add an entry to the cache and then delete it. +Marten.cache.write("foo", "bar", expires_in: 10.minutes) => true +Marten.cache.delete("foo") # => true +``` + +### Incrementing and decrementing values + +If you need to persist integer values that are intended to be incremented or decremented, then you can leverage the [`#increment`](pathname:///api/0.3/Marten/Cache/Store/Base.html#increment(key%3AString%2Camount%3AInt32%3D1%2Cexpires_at%3ATime|Nil%3Dnil%2Cexpires_in%3ATime%3A%3ASpan|Nil%3Dnil%2Cversion%3AInt32|Nil%3Dnil%2Crace_condition_ttl%3ATime%3A%3ASpan|Nil%3Dnil%2Ccompress%3ABool|Nil%3Dnil%2Ccompress_threshold%3AInt32|Nil%3Dnil)%3AInt-instance-method) and [`#decrement`](pathname:///api/0.3/Marten/Cache/Store/Base.html#decrement(key%3AString%2Camount%3AInt32%3D1%2Cexpires_at%3ATime|Nil%3Dnil%2Cexpires_in%3ATime%3A%3ASpan|Nil%3Dnil%2Cversion%3AInt32|Nil%3Dnil%2Crace_condition_ttl%3ATime%3A%3ASpan|Nil%3Dnil%2Ccompress%3ABool|Nil%3Dnil%2Ccompress_threshold%3AInt32|Nil%3Dnil)%3AInt-instance-method) methods. The advantage of doing so is that the increment/decrement operation will be performed in an atomic fashion depending on the cache store you are using (eg. this is the case for the stores provided by the [`marten-memcached-cache`](https://github.com/martenframework/marten-memcached-cache) and [`marten-redis-cache`](https://github.com/martenframework/marten-redis-cache) shards). + +For example: + +```crystal +Marten.cache.increment("mycounter") => 1 +Marten.cache.increment("mycounter", amount: 2) => 3 +Marten.cache.decrement("mycounter") => 2 +``` + +### Clearing the cache + +It's possible to fully clear the content of the cache by leveraging the [`#clear`](pathname:///api/0.3/Marten/Cache/Store/Base.html#clear-instance-method). + +For example: + +```crystal +# Let's add an entry to the cache and then let's clear the cache. +Marten.cache.write("foo", "bar", expires_in: 10.minutes) +Marten.cache.clear +``` + +:::caution +You should be extra careful when using this method because it will fully remove all the entries stored in the cache. Depending on the store implementation, only _namespaced_ entries may be removed (this is the case for the [Redis cache store](https://github.com/martenframework/marten-redis-cache) for example). +::: + +## Template fragment caching + +You can leverage template fragment caching when you want to cache some parts of your [templates](../templates). This capability is enabled by the use of the [`cache`](../templates/reference/tags#cache) template tag. + +This template tag allows caching the content of a template fragment (enclosed within the `{% cache %}...{% endcache %}` tags) for a specific duration. This caching operation is done by leveraging the configured [global cache store](#configuration-and-cache-stores). The tag itself takes at least two arguments: the name to give to the cache fragment and a cache timeout - expressed in seconds. + +For example, the following snippet caches the content enclosed within the `{% cache %}...{% endcache %}` tags for a duration of 3600 seconds and associates it to the "articles" fragment name: + +```html +{% cache "articles" 3600 %} + +{% endcache %} +``` + +It's worth noting that the [`cache`](../templates/reference/tags#cache) template tag allows for the inclusion of additional arguments. These arguments, referred to as "vary on" values, play a crucial role in generating the cache key of the template fragment. Essentially, the cache is invalidated if the value of any of these arguments changes. This feature comes in handy when you need to ensure that the template fragment is cached based on other dynamic values that may impact the generation of the cached content itself. + +For instance, suppose the cached content is dependent on the current locale. In that case, you'd want to make sure that the current locale value is taken into account while caching the template fragment. The ability to pass additional arguments as "vary on" values enables you to achieve precisely that. + +For example: + +```html +{% cache "articles" 3600 current_locale user.id %} + +{% endcache %} +``` + +:::tip +The "key" used for the template fragment cache entry can be a template variable. The same goes for the cache timeout. For example: + +```html +{% cache fragment_name fragment_expiry %} + +{% endcache %} +``` +::: diff --git a/docs/versioned_docs/version-0.3/caching/reference/stores.md b/docs/versioned_docs/version-0.3/caching/reference/stores.md new file mode 100644 index 000000000..2cd685f05 --- /dev/null +++ b/docs/versioned_docs/version-0.3/caching/reference/stores.md @@ -0,0 +1,46 @@ +--- +title: Caching stores +description: Caching stores reference. +sidebar_label: Stores +--- + +## Built-in stores + +### In-memory store + +This is the default store used as part of the [`cache_store`](../../development/reference/settings#cache_store) setting. + +This cache store is implemented as part of the [`Marten::Cache::Store::Memory`](pathname:///api/0.3/Marten/Cache/Store/Memory.html) class. This cache stores all data in memory within the same process, making it a fast and reliable option for caching in single process environments. However, it's worth noting that if you're running multiple instances of your application, the cache data will not be shared between them. + +For example: + +```crystal +Marten.configure do |config| + config.cache_store = Marten::Cache::Store::Memory.new.new(expires_in: 24.hours) +end +``` + +### Null store + +A cache store implementation doesn't store any data. + +This cache store is implemented as part of the [`Marten::Cache::Store::Null`](pathname:///api/0.3/Marten/Cache/Store/Null.html) class. This cache store does not store any data, but provides a way to go through the caching interface. This can be useful in development and testing environments when caching is not desired. + +For example: + +```crystal +Marten.configure do |config| + config.cache_store = Marten::Cache::Store::Null.new.new(expires_in: 24.hours) +end +``` + +## Other stores + +Additional cache stores shards are also maintained under the umbrella of the Marten project or by the community itself and can be used as part of your application depending on your specific caching requirements: + +* [`marten-memcached-cache`](https://github.com/martenframework/marten-memcached-cache) provides a [Memcached](https://memcached.org) cache store +* [`marten-redis-cache`](https://github.com/martenframework/marten-redis-cache) provides a [Redis](https://redis.io) cache store + +:::info +Feel free to contribute to this page and add links to your shards if you've created cache stores that are not listed here! +::: diff --git a/docs/versioned_docs/version-0.3/deployment.mdx b/docs/versioned_docs/version-0.3/deployment.mdx new file mode 100644 index 000000000..f6ecb29b6 --- /dev/null +++ b/docs/versioned_docs/version-0.3/deployment.mdx @@ -0,0 +1,29 @@ +--- +title: Deployment +--- + +import DocCard from '@theme/DocCard'; + +The section explains the steps involved when it comes to deploying a Marten project. It also gives useful tips in order to secure Marten web applications in production. + +## Guides + +
+
+ +
+
+ +## How-to's + +
+
+ +
+
+ +
+
+ +
+
diff --git a/docs/versioned_docs/version-0.3/deployment/how-to/deploy-to-an-ubuntu-server.md b/docs/versioned_docs/version-0.3/deployment/how-to/deploy-to-an-ubuntu-server.md new file mode 100644 index 000000000..feb8706c6 --- /dev/null +++ b/docs/versioned_docs/version-0.3/deployment/how-to/deploy-to-an-ubuntu-server.md @@ -0,0 +1,258 @@ +--- +title: Deploy to an Ubuntu server +description: Learn how to deploy a Marten project to an Ubuntu server. +--- + +This guide covers how to deploy a Marten project to an Ubuntu server. + +## Prerequisites + +To complete the steps in this guide, you will need: + +* an [Ubuntu](https://ubuntu.com) server with SSH access and `sudo` permissions +* a working Marten project +* a domain name targeting your server + +## Install the required dependencies + +The first dependency we want to install on the server is Crystal itself. To do so, you can run the following command: + +```bash +curl -fsSL https://crystal-lang.org/install.sh | sudo bash +``` + +:::tip +Alternatively, you can also refer to [Crystal's official installation instructions](https://crystal-lang.org/install/on_ubuntu/) for Ubuntu if the above command does not work. +::: + +Secondly, we should install a few additional packages that will be required later on: + +* `git` to clone the project's repository +* `nginx` to serve the project's server behind a reverse proxy and also serve [assets](../../assets/introduction) and [media files](../../files/managing-files) +* `postgresql` to handle our database needs + +This can be achieved by running the following command: + +```bash +sudo apt-get install git nginx postgresql +``` + +:::info +This guide assumes the use of [PostgreSQL](https://www.postgresql.org) but can easily be adapted if your project requires another database backend. +::: + +## Create a deployment user + +Let's now create a deployment user. This user will have access to your project's server and will be used to run the application: + +```bash +sudo adduser --disabled-login deploy +``` + +## Create the project folders + +We can now create a deployment folder where we will be able to clone the project repository later on, and store collected assets or media files if necessary. While creating this folder, it is also necessary to ensure that the `deploy` user created previously has access to it: + +```bash +sudo mkdir /srv/ +sudo chown deploy:deploy /srv/ +``` + +## Create a database + +As mentioned previously, this guide assumes the use of [PostgreSQL](https://www.postgresql.org) for the project's database. As such, we need to create a database user and the database itself. To do so, we will need to execute the following commands: + +```bash +su - postgres -c 'createuser deploy' +su - postgres -c 'createdb -O deploy ' +``` + +:::info +PostgreSQL management commands are usually performed as the `postgres` user, hence the use of `su` in the above commands. +::: + +Obviously, you should also ensure that your Marten project is correctly configured to target this database in production. You can have a look at the [database settings](../../development/reference/settings#database-settings) to see what are the available options when it comes to configuring databases. + +## Clone the project + +First, change into the `deploy` user you created previously: + +```bash +su - deploy +``` + +Then you can clone your repository and change into the corresponding folder using the following commands: + +```bash +git clone /srv//project +cd /srv//project +``` + +## Install the dependencies and compile the project + +The next step is to install your project's dependencies. To do so, you can use the [`shards`](https://crystal-lang.org/reference/man/shards/index.html) command as follows: + +```bash +shards install +``` + +We then need to compile the project binary and the [management CLI](../../development/management-commands): + +```bash +crystal build src/server.cr -o bin/server --release +crystal build manage.cr -o bin/manage --release +``` + +The management CLI binary will be helpful in order to [apply migrations](../../models-and-databases/migrations) and to [collect assets](../../development/reference/management-commands#collectassets). + +:::info +Depending on how you are handling assets as part of your projects you may have to perform additional steps. For example, you may have to install Node.js, install additional dependencies, and eventually bundle assets with Webpack if this is applicable to your project! +::: + +## Collect assets + +You will then want to collect your [assets](../../assets/introduction) so that they are uploaded to their final destination. To do so you can leverage the management CLI binary you compiled previously and run the [`collectassets`](../../development/reference/management-commands#collectassets) command: + +```bash +bin/manage collectassets --no-input +``` + +This management command will "collect" all the available assets from the applications' assets directories and from the directories configured in the [`dirs`](../../development/reference/settings#dirs) setting, and ensure that they are "uploaded" to their final destination based on the [assets storage](../../assets/introduction#assets-storage) that is currently configured. + +## Apply the project's migrations + +Then you will want to run your project's [migrations](../../models-and-databases/migrations) to ensure that your models are created at the database level. To achieve this you can leverage the management CLI binary that you compiled in a previous step and run the [`migrate`](../../development/reference/management-commands#migrate) command: + +```bash +bin/manage migrate +``` + +## Setup a SystemD service for your application + +[SystemD](https://systemd.io) is a service manager for Linux that we can leverage in order to easily start or restart our deployed application. As such, we are going to create a service for our application. + +To do so, first ensure that you exit your current shell session as the `deploy` user with `Ctrl-D` or by entering the `exit` command. Then create a service file for your app by typing the following command: + +```bash +nano /etc/systemd/system/.service +``` + +This should open a text editor in your terminal. Copy the following content into it: + +``` +[Unit] +Description= server +After=syslog.target + +[Service] +ExecStart=/srv//project/bin/server +Restart=always +RestartSec=5s +KillSignal=SIGQUIT +WorkingDirectory=/srv//project +Environment="MARTEN_ENV=production" + +[Install] +WantedBy=multi-user.target +``` + +Don't forget to replace the `` placeholders with the right values and, when ready, save the file using `Ctrl-X` and `y`. + +As you can see in the above snippet, we are assuming that the current [Marten environment](../../development/settings#environments) is the production one by setting the `MARTEN_ENV` environment variable to `production`. You should adapt this to your deployment use case obviously. + +:::tip +This service file is also a good place to define any environment variables that may be required by your project's settings. For example, this may be the case for "sensitive" setting values such as the [`secret_key`](../../development/reference/settings#secret_key) setting: as highlighted in [Secure critical setting values](../introduction#secret-key) you could store the value of this setting in a dedicated environment variable and load it from your application's codebase. If you do so, you will also want to add additional lines to the service file in order to define these additional environment variables. For example: + +``` +Environment="MARTEN_SECRET_KEY=" +``` +::: + +In order to ensure that SystemD takes into account the new service you just created, you can then run the following command: + +```bash +systemctl daemon-reload +``` + +And finally, you can start your server with: + +```bash +service start +``` + +Note that in subsequent deployments you will simply want to restart the SystemD service you previously defined. To do so, you can simply use the following command: + +```bash +service restart +``` + +## Setup a Nginx reverse proxy + +Marten project servers are intended to be used behind a reverse proxy such as [Nginx](https://www.nginx.com/) or [Apache](https://httpd.apache.org/). Using a reverse proxy allows you to easily set up an SSL certificate for your server (for example using [Let's Encrypt](https://letsencrypt.org/)), to serve collected assets and media files if applicable, and enhance security and reliability. + +In our case, we will be using [Nginx](https://www.nginx.com/) and create a site configuration for our application. Let's use the following command to do so: + +```bash +nano /etc/nginx/sites-available/.conf +``` + +This should open a text editor in your terminal. Copy the following content into it: + +``` +server { + listen 80; + server_name ; + + include snippets/snakeoil.conf; + + gzip on; + gzip_disable "msie6"; + gzip_vary on; + gzip_proxied any; + gzip_comp_level 6; + gzip_buffers 16 8k; + gzip_http_version 1.1; + gzip_min_length 256; + gzip_types text/plain text/css application/json application/javascript application/x-javascript text/xml application/xml application/xml+rss text/javascript application/vnd.ms-fontobject application/x-font-ttf font/opentype image/svg+xml image/x-icon; + + error_log /var/log/nginx/_error.log; + access_log /var/log/nginx/_access.log; + + location /assets/ { + expires 365d; + alias /; + } + + location /media/ { + alias /; + } + + location / { + proxy_set_header Host $http_host; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; + proxy_redirect off; + proxy_buffering off; + + proxy_pass http://localhost:; + } +} +``` + +Don't forget to replace the ``, ``, ``, ``, and `` placeholders with the right values and, when ready, save the file using `Ctrl-X` and `y`. + +As you can see, the reverse proxy will serve our application on the HTTP port 80 and is configured to target our Marten server host (`localhost`) and port. Because of this, you should ensure that your Marten server is not using the HTTP port 80 (instead it could use something like 8080 or 8000 for example). + +You should also note that the above configuration defines two additional locations in order to serve assets (`/assets/`) and media files (`/media/`). This makes the assumption that those files are _locally_ available on the considered server. As such you should remove these lines if this is not applicable to your use case or if these files are uploaded somewhere else (eg. in a cloud bucket). + +You can then enable this site configuration by creating a symbolic link as follows: + +```bash +ln -s /etc/nginx/sites-available/.conf /etc/nginx/sites-enabled/.conf +``` + +And finally, you can restart the Nginx service with: + +```bash +sudo service nginx restart +``` diff --git a/docs/versioned_docs/version-0.3/deployment/how-to/deploy-to-fly-io.md b/docs/versioned_docs/version-0.3/deployment/how-to/deploy-to-fly-io.md new file mode 100644 index 000000000..7bf4deb6d --- /dev/null +++ b/docs/versioned_docs/version-0.3/deployment/how-to/deploy-to-fly-io.md @@ -0,0 +1,237 @@ +--- +title: Deploy to Fly.io +description: Learn how to deploy a Marten project to Fly.io. +--- + +This guide covers how to deploy a Marten project to [Fly.io](https://fly.io). + +## Prerequisites + +To complete the steps in this guide, you will need: + +* An active account on [Fly.io](https://fly.io). +* The Fly.io CLI [installed](https://fly.io/docs/hands-on/install-flyctl/) and correctly [configured](https://fly.io/docs/getting-started/log-in-to-fly/). +* A functional Marten project. + +## Make your Marten project Fly.io-ready + +Before creating the Fly.io application, it is important to ensure that your project is properly configured for deployment to Fly.io. This section outlines some steps to ensure that your project can be deployed to Fly.io without issues. + +### Create a `Dockerfile` + +We will be deploying our Marten project to Fly.io by leveraging a [Dockerfile strategy](https://fly.io/docs/languages-and-frameworks/dockerfile/). A `Dockerfile` is a text file that contains a set of instructions for building a [Docker](https://www.docker.com/) image. It typically includes a base image, commands to install dependencies, and steps to configure the environment and copy files into the image. + +Your `Dockerfile` should be placed at the root of your project folder and should contain the following content at least: + +```Dockerfile title="Dockerfile" +FROM crystallang/crystal:latest +WORKDIR /app +COPY . . + +ENV MARTEN_ENV=production + +RUN apt-get update +RUN apt-get install -y curl cmake build-essential + +RUN shards install +RUN bin/marten collectassets --no-input +RUN crystal build manage.cr -o bin/manage +RUN crystal build src/server.cr -o bin/server --release + +CMD ["/app/bin/server"] +``` + +As you can see, this Dockerfile builds a Docker image based on the latest version of the Crystal programming language image. It also installs your project's Crystal dependencies, runs the [`collectassets`](../../development/reference/management-commands) management command, and compiles your server's binary. + +It should be noted that this Dockerfile could perform additional operations if needed. For example, some projects may require Node.js in order to install additional dependencies and build your project's assets. This could be achieved with the following additions: + +```Dockerfile title="Dockerfile" +FROM crystallang/crystal:latest +WORKDIR /app +COPY . . + +ENV MARTEN_ENV=production + +RUN apt-get update +RUN apt-get install -y curl cmake build-essential +// highlight-next-line +RUN curl -fsSL https://deb.nodesource.com/setup_16.x | bash - +// highlight-next-line +RUN apt-get install -y nodejs +// highlight-next-line + +// highlight-next-line +RUN npm install +// highlight-next-line +RUN npm run build + +RUN shards install +RUN bin/marten collectassets --no-input +RUN crystal build manage.cr -o bin/manage +RUN crystal build src/server.cr -o bin/server --release + +CMD ["/app/bin/server"] +``` + +### Configure your production server's host and port + +You should ensure that your production server can be accessed from other containers, and on a specific port. To do so, it's important to set the [`host`](../../development/reference/settings#host) setting to `0.0.0.0` and the [`port`](../../development/reference/settings#port) setting to a specific value such as `8000` (which is the port we'll be using throughout this guide). + +This can be achieved by updating your `config/settings/production.cr` production settings file as follows: + +```crystal title="config/settings/production.cr" +Marten.configure :production do |config| + config.host = "0.0.0.0" + config.port = 8000 + + # Other settings... +end +``` + +### Configure key settings from environment variables + +When deploying to Fly.io, you will have to set a few environment variables (later in this guide) that will be used to populate key settings. This should be the case for the [`secret_key`](../../development/reference/settings#secret_key) and [`allowed_hosts`](../../development/reference/settings#allowed_hosts) settings at least. + +As such, it is important to ensure that your project populates these settings by reading their values in corresponding environment variables. This can be achieved by updating your `config/settings/production.cr` production settings file as follows: + +```crystal title="config/settings/production.cr" +Marten.configure :production do |config| + config.secret_key = ENV.fetch("MARTEN_SECRET_KEY", "") + config.allowed_hosts = ENV.fetch("MARTEN_ALLOWED_HOSTS", "").split(",") + + # Other settings... +end +``` + +It should be noted that if your application requires a database, you should also make sure to parse the `DATABASE_URL` environment variable and to configure your [database settings](../../development/reference/settings#database-settings) from the parsed database URL properties. The `DATABASE_URL` variable contains a URL-encoded string that specifies the connection details of your database, such as the database type, hostname, port, username, password, and database name. + +This can be accomplished as follows for a PostgreSQL database: + +```crystal title="config/settings/production.cr" +Marten.configure :production do |config| + if ENV.has_key?("DATABASE_URL") + # Note: DATABASE_URL isn't available at build time... + config.database do |db| + database_uri = URI.parse(ENV.fetch("DATABASE_URL")) + + db.backend = :postgresql + db.host = database_uri.host + db.port = database_uri.port + db.user = database_uri.user + db.password = database_uri.password + db.name = database_uri.path[1..] + + # Fly.io's Postgres works over an internal & encrypted network which does not support SSL. + # Hence, SSL must be disabled. + db.options = {"sslmode" => "disable"} + end + end + + # Other settings... +end +``` + +### Optional: set up the asset serving middleware + +In order to easily serve your application's assets in Fly.io, you can make use of the [`Marten::Middleware::AssetServing`](../../handlers-and-http/reference/middlewares#asset-serving-middleware) middleware. Indeed, it won't be possible to configure a web server such as [Nginx](https://nginx.org) to serve your assets directly on Fly.io if you intend to use a "local file system" asset store (such as [`Marten::Core::Store::FileSystem`](pathname:///api/0.3/Marten/Core/Storage/FileSystem.html)). + +To palliate this, you can make use of the [`Marten::Middleware::AssetServing`](../../handlers-and-http/reference/middlewares#asset-serving-middleware) middleware. Obviously, this is not necessary if you intend to leverage a cloud storage provider (like Amazon's S3 or GCS) to store and serve your collected assets (in this case, you can simply skip this section). + +In order to use this middleware, you can "insert" the corresponding class at the beginning of the [`middleware`](../../development/reference/settings#middleware) setting when defining production settings. For example: + +```crystal +Marten.configure :production do |config| + config.middleware.unshift(Marten::Middleware::AssetServing) + + # Other settings... +end +``` + +The middleware will serve the collected assets available under the assets root ([`assets.root`](../../development/reference/settings#root) setting). It is also important to note that the [`assets.url`](../../development/reference/settings#url) setting must align with the Marten application domain or correspond to a relative URL path (e.g., `/assets/`) for this middleware to work correctly. + +## Create the Fly.io app + +To begin, the initial action required is to generate your Fly.io application itself. This can be achieved by executing the `fly launch` command as follows: + +```bash +fly launch --no-deploy --no-cache --internal-port 8000 --name --env MARTEN_ALLOWED_HOSTS=.fly.dev +``` + +The above command creates a Fly.io application whose internal port is set to `8000` while also ensuring that the `MARTEN_ALLOWED_HOSTS` environment variable is set to your future app domain. The command will ask you to choose a specific [region](https://fly.io/docs/reference/regions/) for your application and will create a `fly.toml` file whose content should look like this: + +```toml title="fly.toml" +app = "" +primary_region = "" + +[env] + MARTEN_ALLOWED_HOSTS = ".fly.dev" + +[http_service] + internal_port = 8000 + force_https = true + auto_stop_machines = true + auto_start_machines = true +``` + +The `fly.toml` is a configuration file used by Fly.io to know how to deploy your application to the Fly.io platform. + +:::info +In this guide, the `` placeholder refers to the Fly.io application name that you have chosen for your project. You should replace `` with the actual name of your application in all the relevant commands and code snippets mentioned in this guide. +::: + +## Set up environment secrets + +It is recommended to define the `MARTEN_SECRET_KEY` environment variable order to populate the [`secret_key`](../../development/reference/settings#secret_key) setting, as mentioned in [Configure key settings from environment variables](#configure-key-settings-from-environment-variables). + +Fly.io gives the ability to define such sensitive setting values using [runtime secrets](https://fly.io/docs/reference/secrets/). In this light, we can create a `MARTEN_SECRET_KEY` secret by using the `fly secrets` command as follows: + +```bash +fly secrets set MARTEN_SECRET_KEY=$(openssl rand -hex 16) +``` + +## Set up a database + +You'll need to provision a Fly.io PostgreSQL database if your application makes use of models and migrations (otherwise you can skip this step!). + +In this light, you first need to create a PostgreSQL cluster with the following command: + +```bash +fly pg create --name -db +``` + +Then you will need to "attach" the PostgreSQL cluster you just created with your actual application. This can be achieved with the following command: + +```bash +fly postgres attach -db --app +``` + +Additionally, you will want to ensure that migrations are automatically applied every time your project is deployed. To do so, you can update the `fly.toml` file that was generated previously and add the following section to it: + +```toml title="fly.toml" +app = "" +primary_region = "" + +[env] + MARTEN_ALLOWED_HOSTS = ".fly.dev" + +[http_service] + internal_port = 8000 + force_https = true + auto_stop_machines = true + auto_start_machines = true + +// highlight-next-line +[deploy] +// highlight-next-line + release_command = "bin/manage migrate" +``` + +## Deploy the application + +The final step is to upload your application's code to Fly.io. This can be done by using the following command: + +```bash +fly deploy +``` + +It is worth mentioning that a few things will happen when you push your application's code to Fly.io like in the above example. Indeed, Fly.io will build a Docker image of your application based on the `Dockerfile` you defined [previously](#create-a-dockerfile) and then push it to the Fly.io registry (a private Docker registry maintained by Fly.io). Once this is done, it will launch a Docker container by using the obtained Docker image, and then route incoming traffic to the running container. diff --git a/docs/versioned_docs/version-0.3/deployment/how-to/deploy-to-heroku.md b/docs/versioned_docs/version-0.3/deployment/how-to/deploy-to-heroku.md new file mode 100644 index 000000000..d1819b6e3 --- /dev/null +++ b/docs/versioned_docs/version-0.3/deployment/how-to/deploy-to-heroku.md @@ -0,0 +1,215 @@ +--- +title: Deploy to Heroku +description: Learn how to deploy a Marten project to Heroku. +--- + +This guide covers how to deploy a Marten project to [Heroku](https://heroku.com). + +## Prerequisites + +To complete the steps in this guide, you will need: + +* An active account on [Heroku](https://heroku.com). +* The [Heroku CLI](https://devcenter.heroku.com/articles/heroku-cli) installed and correctly configured. +* A functional Marten project. + +## Make your Marten project Heroku-ready + +Before creating the Heroku application, it is important to ensure that your project is properly configured for deployment to Heroku. This section outlines some necessary steps to ensure that your project can be deployed to Heroku without issues. + +### Create a `Procfile` + +You should first ensure that your project defines a [`Procfile`](https://devcenter.heroku.com/articles/procfile), at the root of your project folder. A Procfile specifies the commands Heroku's dynos run, defining process types like web servers and background workers. + +Your Procfile should contain the following content at least: + +```procfile title="Procfile" +web: bin/server --port \$PORT +``` + +:::info +The `PORT` environment variable is automatically defined by Heroku. That's why we have to ensure that its value is forwarded to your server. +::: + +If your application requires the use of a database, you should also add a [release process](https://devcenter.heroku.com/articles/procfile#the-release-process-type) that runs the [`migrate`](../../development/reference/management-commands#migrate) management command to your Procfile: + +```procfile title="Procfile" +web: bin/server --port \$PORT +release: marten migrate +``` + +This will ensure that your database is properly migrated during each deployment. + +### Configure the root path + +During deployment on Heroku, your application is prepared and compiled in a temporary directory, which is distinct from the location where the server runs your application. Specifically, the root of your application will be available under the `/app` folder on the Heroku platform. It's important to keep this in mind when setting up your application for deployment to Heroku. + +Marten's [application mechanism](../../development/applications) relies heaviliy on paths when it comes to locate things like [templates](../../templates), [translations](../../i18n), or [assets](../../assets). Because the path where your application is compiled will differ from the path where it runs, we need to ensure that you explicitly configure Marten so that it can find your project structure. + +To address this, we need to define a specific "root path" for your project in production. The root path specifies the actual location of the project sources in your system. This can prove helpful in scenarios where the project was compiled in a specific location different from the final destination where the project sources (and the `lib` folder) are copied, which is the case with Heroku. + +In this light, we can set the [`root_path`](../../development/reference/settings#root_path) setting to `/app` as follows: + +```crystal title="config/settings/production.cr" +Marten.configure :production do |config| + config.root_path = "/app" + + # Other settings... +end +``` + +As highlighted in the above example, this should be done in your "production" settings file. + +### Configure key settings from environment variables + +When deploying to Heroku, you will have to set a few environment variables (later in this guide) that will be used to populate key settings. This should be the case for the [`secret_key`](../../development/reference/settings#secret_key) and [`allowed_hosts`](../../development/reference/settings#allowed_hosts) settings at least. + +As such, it is important to ensure that your project populates these settings by reading their values in corresponding environment variables. This can be achieved by updating your `config/settings/production.cr` production settings file as follows: + +```crystal title="config/settings/production.cr" +Marten.configure :production do |config| + config.secret_key = ENV.fetch("MARTEN_SECRET_KEY") + config.allowed_hosts = ENV.fetch("MARTEN_ALLOWED_HOSTS", "").split(",") + + # Other settings... +end +``` + +It should be noted that if your application requires a database, you should also make sure to parse the `DATABASE_URL` environment variable and to configure your [database settings](../../development/reference/settings#database-settings) from the parsed database URL properties. The `DATABASE_URL` variable contains a URL-encoded string that specifies the connection details of your database, such as the database type, hostname, port, username, password, and database name. + +This can be accomplished as follows for a PostgreSQL database: + +```crystal title="config/settings/production.cr" +Marten.configure :production do |config| + config.database do |db| + database_uri = URI.parse(ENV.fetch("DATABASE_URL")) + + db.backend = :postgresql + db.host = database_uri.host + db.port = database_uri.port + db.user = database_uri.user + db.password = database_uri.password + db.name = database_uri.path[1..] + end + + # Other settings... +end +``` + +### Optional: set up the asset serving middleware + +In order to easily serve your application's assets in Heroku, you can make use of the [`Marten::Middleware::AssetServing`](../../handlers-and-http/reference/middlewares#asset-serving-middleware) middleware. Indeed, it won't be possible to configure a web server such as [Nginx](https://nginx.org) to serve your assets directly on Heroku if you intend to use a "local file system" asset store (such as [`Marten::Core::Store::FileSystem`](pathname:///api/0.3/Marten/Core/Storage/FileSystem.html)). + +To palliate this, you can make use of the [`Marten::Middleware::AssetServing`](../../handlers-and-http/reference/middlewares#asset-serving-middleware) middleware. Obviously, this is not necessary if you intend to leverage a cloud storage provider (like Amazon's S3 or GCS) to store and serve your collected assets (in this case, you can simply skip this section). + +In order to use this middleware, you can "insert" the corresponding class at the beginning of the [`middleware`](../../development/reference/settings#middleware) setting when defining production settings. For example: + +```crystal +Marten.configure :production do |config| + config.middleware.unshift(Marten::Middleware::AssetServing) + + # Other settings... +end +``` + +The middleware will serve the collected assets available under the assets root ([`assets.root`](../../development/reference/settings#root) setting). It is also important to note that the [`assets.url`](../../development/reference/settings#url) setting must align with the Marten application domain or correspond to a relative URL path (e.g., `/assets/`) for this middleware to work correctly. + +## Create the Heroku app + +To begin, the initial action required is to generate your Heroku application itself. This can be achieved by executing the `heroku create` command as follows: + +```bash +heroku create +``` + +:::info +In this guide, the `` placeholder refers to the Heroku application name that you have chosen for your project. You should replace `` with the actual name of your application in all the relevant commands and code snippets mentioned in this guide. +::: + +## Set up the required buildpacks + +Heroku leverages [buildpacks](https://devcenter.heroku.com/articles/buildpacks) in order to "compile" web applications (which can include installing dependencies, compiling actual binaries, etc). In the context of a Marten project, it is recommended to use two buildbacks: + +1. First, the [Node.js official buildback](https://github.com/heroku/heroku-buildpack-nodejs) in order to "build" your project's assets. +2. Second, the [Marten official buildback](https://github.com/martenframework/heroku-buildpack-marten) in order to (i) compile your server's binary, (ii) compile the Marten CLI, and (iii) [collect assets](../../assets/introduction). + +The sequence of buildpacks applied during the deployment process is critical: the Node.js buildpack must be the first one applied, to ensure that Heroku can initiate the necessary Node.js installations and configurations. This approach will guarantee that when the Marten buildpack is activated, the assets will have already been created and are ready to be "collected" through the [`collectassets`](../../development/reference/management-commands#collectassets) management command. + +You can ensure that these buildpacks are used by running the following commands: + +```bash +heroku buildpacks:add heroku/nodejs +heroku buildpacks:add https://github.com/martenframework/heroku-buildpack-marten +``` + +:::tip +It is important to mention that the use of the [Node.js official buildback](https://github.com/heroku/heroku-buildpack-nodejs) is completely optional: you should only use it if your project leverages Node.js to build some assets. +::: + +## Set up environment variables + +### `MARTEN_ENV` + +At least one environment variable needs to be configured in order to ensure that your Marten project operates in production mode in Heroku: the `MARTEN_ENV` variable. This variable determines the current environments (and the associated settings to apply). + +To set this environment variable, you can leverage the `heroku config:set` command as follows: + +```bash +heroku config:set MARTEN_ENV=production +``` + +### `MARTEN_SECRET_KEY` + +It is also recommended to define the `MARTEN_SECRET_KEY` environment variable in order to populate the [`secret_key`](../../development/reference/settings#secret_key) setting, as mentioned in [Configure key settings from environment variables](#configure-key-settings-from-environment-variables). + +To set this environment variable, you can leverage the `heroku config:set` command as follows: + +```bash +heroku config:set MARTEN_SECRET_KEY=$(openssl rand -hex 16) +``` + +### `MARTEN_ALLOWED_HOSTS` + +Finally, we want to ensure that the [`allowed_hosts`](../../development/reference/settings#allowed_hosts) setting contains the actual domain of your Heroku application, which is required as part of Marten's [HTTP Host Header Attacks Protection mechanism](../../security/introduction#http-host-header-attacks-protection). + +To set this environment variable, you can use the following command: + +```bash +heroku config:set MARTEN_ALLOWED_HOSTS=.herokuapp.com +``` + +## Set up a database + +You'll need to provision a Heroku PostgreSQL database if your application makes use of models and migrations. To do so, you can make use of the following command: + +```bash +heroku addons:create heroku-postgresql:mini +``` + +:::info +You should replace `mini` in the above command by your desired [Postgres plan](https://devcenter.heroku.com/articles/heroku-postgres-plans). +::: + +## Upload the application + +The final step is to upload your application's code to Heroku. This can be done by using the standard `git push` command to copy the local `main` branch to the `main` branch on Heroku: + +```bash +git push heroku main +``` + +It is worth mentioning that a few things will happen when you push your application's code to Heroku like in the above example. Indeed, Heroku will detect the type of your application and apply the buildpacks you [configured previously](#set-up-the-required-buildpacks) (ie. first the Node.js one and then the Marten one). As part of this step, your application's dependencies will be installed and your project will be compiled. If you defined a `release` process in your `Procfile` (like explained in [Create a Procfile](#create-a-procfile)), the specfied command will also be executed (for example in order to run your project's migrations). + +A few additional things should also be noted: + +* The compiled server binary will be placed under the `bin/server` path. +* Your project's `manage.cr` file will be compiled as well and will be available by simply calling the `marten` command. This means that you can run `marten ` if you need to call specific [management commands](../../development/management-commands). +* The Marten buildpack will automatically call the [`collectassets`](../../development/reference/management-commands#collectassets) management command in order to collect your project's [assets](../../assets/introduction) and copy them to your configured assets storage. You can set the `DISABLE_COLLECTASSETS` environment variable to `1` if you don't want this behavior. + +## Run management commands + +If you need to run additional [management commands](../../development/management-commands) in your provisioned application, you can use the `heroku run` command. For instance: + +```bash +heroku run marten listmigrations +``` diff --git a/docs/versioned_docs/version-0.3/deployment/introduction.md b/docs/versioned_docs/version-0.3/deployment/introduction.md new file mode 100644 index 000000000..b3f988545 --- /dev/null +++ b/docs/versioned_docs/version-0.3/deployment/introduction.md @@ -0,0 +1,122 @@ +--- +title: Deploying Marten projects +description: Learn about the things to consider when deploying Marten web applications. +sidebar_label: Introduction +--- + +This section describes what's involved when it comes to deploying a Marten web application and highlights some important things to consider before performing deploys. + +## Overview + +Each deployment pipeline is unique and will vary from one project to another. That being said, a few things and requirements will be commonly encountered when it comes to deploying a Marten project: + +1. installing your project's dependencies +2. compiling your project's server and [management CLI](../development/management-commands) +3. collecting your project's [assets](../assets/introduction) +4. applying any pending migrations to your database +5. starting the compiled server + +Where, when, and how these steps are performed will vary from one project to another. Each of these steps is highlighted below along with some recommendations. + +It should also be noted that a few guides highlighting common deployment strategies can be leveraged if necessary: + +* [Deploying to an Ubuntu server](./how-to/deploy-to-an-ubuntu-server) + +### Installing dependencies + +One of the first things you need to do when deploying a Marten project is to ensure that the dependencies of the projects are available. In this light, you can leverage the `shards install` command to install your project's Crystal dependencies (assuming that Crystal is installed on your destination machine). Obviously, your project may require the installation of other types of dependencies (such as Node.js dependencies for example), which you have to take care of as well. + +### Compiling your project + +Your project server and [management CLI](../development/management-commands) need to be compiled to run your project's server and to execute additional deployment-related management commands (eg. to [collect assets](#collecting-assets) or [apply migrations](#applying-migrations)). + +When it comes to your project server, you will usually need to compile the `src/server.cr` file (which is automatically created when generating new projects via the [`new`](../development/reference/management-commands#new) management command). This can be achieved with the following command: + +```bash +crystal build src/server.cr -o bin/server --release +``` + +:::tip +In the above example, the server binary is compiled using the `-o bin/server` option, which ensures that the compiled binary will be named `server` and stored under the related `bin` folder. You should obviously adapt this to your production environment. +::: + +The [management CLI](../development/management-commands) is provided by the `manage.cr` file located at the root of your project. As usual, this file is also automatically generated for you when creating Marten projects through the use of the [`new`](../development/reference/management-commands#new) management command. Compiling this binary can be done with the following command: + +```bash +crystal build manage.cr -o bin/manage --release +``` + +:::info +The above compilation commands make use of the `--release` flag, which enables compiler optimizations. As a result, the compilation of the final binaries may take quite some time depending on your project. You can also avoid using `--release` if needed but technically performances could be impacted. See [Release builds](https://crystal-lang.org/reference/man/crystal/index.html#release-builds) for more details on this subject. +::: + +### Collecting assets + +You need to ensure that your project and applications assets (eg. JavaScripts, CSS files, etc) are "collected" at deploy time so that they are placed at the final destination from which they will be served: this operation is made available through the use of the [`collectassets`](../development/reference/management-commands#collectassets) management command. This "destination" depends on your deployment strategy and your configured [assets settings](../development/reference/settings#assets-settings): it can be as simple as moving all these assets to a dedicated folder in your server (so that they can be served by your web server), or it can involve uploading these assets to an S3 or GCS bucket for example. + +In order to collect assets at deploy time, you will want to use the compiled `manage` binary and run the [`collectassets`](../development/reference/management-commands#collectassets) management command (as mentioned previously) with the `--no-input` flag set in order to disable user prompts: + +```bash +bin/manage collectassets --no-input +``` + +:::info +The assets handling documentation also provides a few [guidelines](../assets/introduction#serving-assets-in-production) on how to serve asset files in production that may be worth reading. +::: + +### Applying migrations + +Your projects will likely make use of models, which means that you will need to ensure that those are properly created at your configured database level by running the associated migrations. + +To do so, you can use the compiled `manage` binary and run the [`migrate`](../development/reference/management-commands#migrate) management command: + +```bash +bin/manage migrate +``` + +Please refer to [Migrations](../models-and-databases/migrations) to learn more about model migrations. + +### Running the server + +You can run the compiled Marten server using the following command (obviously the location of the binary depends on [how the compilation was actually performed](#compiling-your-project)): + +```bash +bin/server +``` + +It's important to note that the Marten server is intended to be used behind a reverse proxy such as [Nginx](https://www.nginx.com/) or [Apache](https://httpd.apache.org/): you will usually want to configure such reverse proxy so that it targets your configured Marten server host and port. In this light, you should ensure that your Marten server is not using the HTTP port 80 (instead it could use something like 8080 or 8000 for example). + +Depending on your use cases, a reverse proxy will also allow you to easily serve other contents such as [assets](../assets/introduction) or [uploaded files](../files/managing-files), and to use SSL/TLS. + +:::tip +It is possible to run multiple processes of the same server behind a reverse proxy such as Nginx. Indeed, each compiled server can accept optional parameters to override the host and/or port being used. These parameters are respectively `--bind` (or `-b`) and `--port` (or `-p`). For example: + +```bash +bin/server -b 127.0.0.1 +bin/server -p 8080 +``` +::: + +## Additional tips + +This section lists a few additional things to consider when deploying Marten projects. + +### Secure critical setting values + +You should pay attention to the value of some of your settings in production environments. + +#### Debug mode + +You should ensure that the [`debug`](../development/reference/settings#debug) setting is always set to `false` in production environments. Indeed, the debug mode can help for development purposes because it outputs useful tracebacks and site-related information. But there is a risk that all this information leaks somewhere if you enable this mode in production. + +#### Secret key + +You should ensure that the value of the [`secret_key`](../development/reference/settings#secret_key) setting is not hardcoded in your [config files](../development/settings). Indeed, this setting value must be kept secret and you should ensure that it's loaded dynamically instead. For example, this setting's value could be set in a dedicated environment variable (or dotenv file) and loaded as follows: + +```crystal +Marten.configure do |config| + config.secret_key = ENV.fetch("MARTEN_SECRET_KEY") { raise "Missing MARTEN_SECRET_KEY env variable" } + + # [...] +end +``` diff --git a/docs/versioned_docs/version-0.3/development.mdx b/docs/versioned_docs/version-0.3/development.mdx new file mode 100644 index 000000000..e8135349c --- /dev/null +++ b/docs/versioned_docs/version-0.3/development.mdx @@ -0,0 +1,43 @@ +--- +title: Development +--- + +import DocCard from '@theme/DocCard'; + +Marten provides various tools and mechanisms that you can leverage in order to develop, structure, and test projects and applications. + +## Guides + +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +## How-to's + +
+
+ +
+
+ +## Reference + +
+
+ +
+
+ +
+
diff --git a/docs/versioned_docs/version-0.3/development/applications.md b/docs/versioned_docs/version-0.3/development/applications.md new file mode 100644 index 000000000..761b94099 --- /dev/null +++ b/docs/versioned_docs/version-0.3/development/applications.md @@ -0,0 +1,119 @@ +--- +title: Applications +description: Learn how to leverage applications to structure your projects. +sidebar_label: Applications +--- + +Marten projects can be organized into logical and reusable components called "applications". These applications can contribute specific behaviors and abstractions to a project, including [models](../models-and-databases), [handlers](../handlers-and-http), and [templates](../templates). They can be packaged and reused across various projects as well. + +## Overview + +A Marten **application** is a set of abstractions (defined under a dedicated and unique folder) that provides some set of features. These abstractions can correspond to [models](../models-and-databases), [handlers](../handlers-and-http), [templates](../templates), [schemas](../schemas/), etc. + +Marten projects always use one or many applications. Indeed, each Marten project comes with a default [main application](#the-main-application) that corresponds to the standard `src` folder: models, migrations, or other classes defined in this folder are associated with the main application by default (unless they are part of another _explicitly defined_ application). As projects grow in size and scope, it is generally encouraged to start thinking in terms of applications and how to split models, handlers, or features across multiple apps depending on their intended responsibilities. + +Another benefit of applications is that they can be packaged and reused across multiple projects. This allows third-party libraries and shards to easily contribute models, migrations, handlers, or templates to other projects. + +## Using applications + +The use of applications must be manually enabled within projects: this is done through the use of the [`installed_apps`](./reference/settings#installedapps) setting. + +This setting corresponds to an array of installed app classes. Indeed, each Marten application must define a subclass of [`Marten::App`](pathname:///api/0.3/Marten/App.html) to specify a few things such as the application label (see [Creating applications](#creating-applications) for more information about this). When those subclasses are specified in the `installed_apps` setting, the applications' models, migrations, assets, and templates will be made available to the considered project. + +For example: + +```crystal +Marten.configure do |config| + config.installed_apps = [ + FooApp, + BarApp, + ] +end +``` + +Adding an application class inside this array will have the following impact on the considered project: + +* the models of this application and the associated migrations will be used +* the templates of the application will be made available to the templates engine +* the assets of the application will be made available to the assets engine +* the management commands defined by the application will be made available to the Marten CLI + +### The main application + +The "main" application is a default application that is always implicitly used by Marten projects (which means that it does not appear in the [`installed_apps`](./reference/settings#installedapps) setting). This application is associated with the standard `src` folder: this means that models, migrations, assets, or templates defined in this folder will be associated with the main application by default. For example, models defined under a `src/models` folder would be associated with the main application. + +:::info +The main application is associated with the `main` label. This means that models of the main application that do not define an explicit table name will have table names starting with `main_`. +::: + +It should be noted that it is possible to create _explicitly defined_ applications whose structures live under the `src` folder as well: the abstractions (eg. models, migrations, etc) of these applications will be associated with them and _not_ with the main application. This is because abstractions are always associated with the closest application in the files/folders structure. + +In the end, the main application provides a convenient way for starting projects and prototyping without requiring to spec out how projects will be organized in terms of applications upfront. That being said, as projects grow in size and scope, it is really encouraged to start thinking in terms of applications and how to split abstractions and features across multiple apps depending on their intended responsibilities. + +### Order of installed applications + +You should note that the order in which installed applications are defined in the [`installed_apps`](./reference/settings#installedapps) setting can actually matter. + +For example, a "foo" app might define a `test.html` template, and a similar template with the exact same name might be defined by a "bar" app. If the "foo" app appears before the "bar" app in the array of installed apps, then requesting and rendering the `test.html` template will actually involve the "foo" app's template only. This is because template loaders associated with app directories iterate over applications in the order in which they are defined in the installed apps array. + +This is why it is always important to _namespace_ abstractions, assets, templates, and locales when creating applications. Failing to do so exposes apps to conflicts with other applications' code. As such, in the previous example, the "foo" app should've defined a `foo/test.html` template while the "bar" app should've defined a `bar/test.html` template to avoid possible conflicts. + +## Creating applications + +Creating applications can be done very easily through the use of the [`new`](./reference/management-commands#new) management command. For example: + +```bash +marten new app blog --dir=src/blog +``` + +Running such a command will usually create the following directory structure: + +``` +src/blog +├── handlers +├── migrations +├── models +├── schemas +├── templates +├── app.cr +└── cli.cr +``` + +These files and folders are described below: + +| Path | Description | +| ----------- | ----------- | +| handlers/ | Empty directory where the request handlers of the application will be defined. | +| migrations/ | Empty directory that will store the migrations that will be generated for the models of the application. | +| models/ | Empty directory where the models of the application will be defined. | +| schemas/ | Empty directory where the schemas of the application will be defined. | +| templates/ | Empty directory where the templates of the application will be defined. | +| app.cr | Definition of the application configuration abstraction; this is also where application files requirements should be defined. | +| cli.cr | Requirements of CLI-related files, such as migrations for example. | + +The most important file of an application is the `app.cr` one. This file usually includes all the app requirements and defines the application configuration class itself, which must be a subclass of the [`Marten::App`](pathname:///api/0.3/Marten/App.html) abstract class. This class allows mainly to define the "label" identifier of the application (through the use of the [`#label`](pathname:///api/0.3/Marten/Apps/Config.html#label(label%3AString|Symbol)-class-method) class method): this identifier must be unique across all the installed applications of a project and is used to generate things like model table names or migration classes. + +Here is an example `app.cr` file content for a hypothetic "blog" app: + +```crystal +require "./handlers/**" +require "./models/**" +require "./schemas/**" + +module Blog + class App < Marten::App + label "blog" + end +end +``` + +:::info +_Where_ the `app.cr` file is located is important: the directory where this file is defined is also the directory where key folders like `models`, `migrations`, `templates`, etc, must be present. This is necessary to ensure that these files and abstractions are associated with the considered app. +::: + +Another very important file is the `cli.cr` one: this file is there to define all the CLI-related requirements and will usually be required directly by your project's `manage.cr` file. _A minima_ the `cli.cr` file should require model migrations, but it could also require the management commands provided by the application. For example: + +```crystal +require "./cli/**" +require "./migrations/**" +``` diff --git a/docs/versioned_docs/version-0.3/development/how-to/create-custom-commands.md b/docs/versioned_docs/version-0.3/development/how-to/create-custom-commands.md new file mode 100644 index 000000000..098b2b433 --- /dev/null +++ b/docs/versioned_docs/version-0.3/development/how-to/create-custom-commands.md @@ -0,0 +1,205 @@ +--- +title: Create custom commands +description: How to create custom management commands. +--- + +Marten lets you create custom management commands as part of your [applications](../applications). This allows you to contribute new features and behaviors to the Marten CLI. + +## Basic management command definition + +Custom management commands are defined as subclasses of the [`Marten::CLI::Command`](pathname:///api/0.3/Marten/CLI/Manage/Command/Base.html) abstract class. Such subclasses should be defined in a `cli/` folder at the root of the application, and it should be ensured that they are required by your `cli.cr` file (see [Creating applications](../applications#creating-applications) for more details regarding the structure of an application). + +Management command classes must at least define a [`#run`](pathname:///api/0.3/Marten/CLI/Manage/Command/Base.html#run-instance-method) method, which will be called when the subcommand is executed: + +```crystal +class MyCommand < Marten::CLI::Command + help "Command that does something" + + def run + # Do something + end +end +``` + +As you can see in the previous example, the [`#help`](pathname:///api/0.3/Marten/CLI/Manage/Command/Base.html#help(help%3AString)-class-method) class method allows setting a "help text" that will be displayed when the help information of the command is requested. + +If the above command was part of an installed application, it could be executed by using the Marten CLI as follows: + +```bash +marten my_command +``` + +## Accepting options and arguments + +Marten management commands can accept options and arguments. These differ and may be used for different use cases: + +* options usually use the `-h` / `--help` style and can receive arguments if needed. They can be specified in any order +* arguments are _positional_ and only their values must be specified + +By default options and arguments are always optional. That being said, they can be made mandatory in the command execution logic if needed. + +Both options and arguments must be specified in the optional [`#setup`](pathname:///api/0.3/Marten/CLI/Manage/Command/Base.html#setup-instance-method) method: this method will be called to prepare the definition of the command, including its arguments and options. + +For example: + +```crystal +class MyCommand < Marten::CLI::Command + help "Command that does something" + + @arg1 : String? + @example : Bool = false + + def setup + on_argument(:arg1, "The first argument") { |v| @arg1 = v } + on_option("example", "An example option") { @example = true } + end + + def run + # Do something + end +end +``` + +In the above example, the [`#on_argument`](pathname:///api/0.3/Marten/CLI/Manage/Command/Base.html#on_argument(name%3AString|Symbol%2Cdescription%3AString%2C%26block%3AString->)-instance-method) instance method is used to define an `arg1` argument. This method requires an argument name, an associated help text, and a proc where the value of the argument will be forwarded at execution time (which allows you to assign it to an instance variable or process it if you wish to). Similarly, the [`#on_option`](pathname:///api/0.3/Marten/CLI/Manage/Command/Base.html#on_option(flag%3AString|Symbol%2Cdescription%3AString%2C%26block%3AString->)-instance-method) instance method is used to define an `example` option. In this case, the name of the option and its associated help text must be specified, and a proc can be defined to identify that the option was specified at execution time (which can be used to set a related boolean instance variable for example). + +The above command would produce the following help information: + +``` +Usage: marten my_command [options] [arg1] + +Command that does something + +Arguments: + arg1 The first argument + +Options: + --example An example option + --error-trace Show full error trace (if a compilation is involved) + --no-color Disable colored output + -h, --help Show this help +``` + +### Configuring options + +As mentioned previously, it is possible to make use of the [`#on_option`](pathname:///api/0.3/Marten/CLI/Manage/Command/Base.html#on_option(flag%3AString|Symbol%2Cdescription%3AString%2C%26block%3AString->)-instance-method) instance method to configure a specific command option (eg. `--option`). It expects a flag name and a description, and it yields a block to let the command properly assign the option value to the command object at execution time: + +```crystal +on_option("example", "An example option") { @example = true } +``` + +Note that the `--` must not be included in the option name. + +Alternatively, it is possible to specify options that accept both a short flag (eg. `-h`) and a long flag (eg. `--help`): + +```crystal +on_option("e", "example", "An example option") { @example = true } +``` + +### Configuring options that accept arguments + +It is possible to make use of the [`#on_option_with_arg`](pathname:///api/0.3/Marten/CLI/Manage/Command/Base.html#on_option_with_arg(flag%3AString|Symbol%2Carg%3AString|Symbol%2Cdescription%3AString%2C%26block%3AString->)-instance-method) instance method to configure a specific command option with an associated argument. This method will configure a command option (eg. `--option`) and an associated argument. It expects a flag name, an argument name, and a description. It yields a block to let the command properly assign the option to the command object at execution time: + +```crystal +on_option_with_arg(:option, :arg, "The name of the option") { @arg = arg } +``` + +Alternatively, it is possible to specify options that accept both a short flag (eg. `-h`) and a long flag (eg. `--help`): + +```crystal +on_option_with_arg("o", "option", "arg", "The name of the option") { |arg| @arg = arg } +``` + +### Configuring arguments + +As mentioned previously, it is possible to make use of the [`#on_argument`](pathname:///api/0.3/Marten/CLI/Manage/Command/Base.html#on_argument(name%3AString|Symbol%2Cdescription%3AString%2C%26block%3AString->)-instance-method) instance method in order to configure a specific command argument. This method expects an argument name and a description, and it yields a block to let the command properly assign the argument value to the command object at execution time: + +```crystal +on_argument(:arg, "The name of the argument") { |value| @arg_var = value } +``` + +:::caution +It should be noted that the order in which arguments are defined is important: this order corresponds to the order in which arguments will need to be specified when invoking the subcommand. +::: + +## Outputting text contents + +When writing management commands, you will likely need to write text contents to the output file descriptor. To do so, you can make use of the [`#print`](pathname:///api/0.3/Marten/CLI/Manage/Command/Base.html#print(msg%2Cending%3D"\n")-instance-method) instance method: + +```crystal +class HelloWorldCommand < Marten::CLI::Command + help "Command that prints Hello World!" + + def run + print("Hello World!") + end +end +``` + +It should be noted that you can also choose to "style" the content you specify to [`#print`](pathname:///api/0.3/Marten/CLI/Manage/Command/Base.html#print(msg%2Cending%3D"\n")-instance-method) by wrapping your string with a call to the [`#style`](pathname:///api/0.3/Marten/CLI/Manage/Command/Base.html#style(msg%2Cfore%3Dnil%2Cmode%3Dnil)-instance-method) method. For example: + +```crystal +class HelloWorldCommand < Marten::CLI::Command + help "Command that prints Hello World!" + + def run + print(style("Hello World!", fore: :light_blue, mode: :bold)) + end +end +``` + +As you can see, the [`#style`](pathname:///api/0.3/Marten/CLI/Manage/Command/Base.html#style(msg%2Cfore%3Dnil%2Cmode%3Dnil)-instance-method) method can be used to apply `fore` and `mode` styles to a specific text value. The values you can use for the `fore` and `mode` arguments are the same as the ones that you can use with the [`Colorize`](https://crystal-lang.org/api/Colorize.html) module (which comes with the standard library). + +## Handling error cases + +You will likely want to handle error situations when writing management commands. For example, to return error messages if a specified argument is not provided or if it is invalid. To do so you can make use of the [`#print_error`](pathname:///api/0.3/Marten/CLI/Manage/Command/Base.html#print_error(msg)-instance-method) helper method, which will print the passed string to the error file descriptor: + +```crystal +class HelloWorldCommand < Marten::CLI::Command + help "Command that prints Hello World!" + + @name : String? + + def setup + on_argument(:name, "A name") { |v| @name = v } + end + + def run + if @name.nil? + print_error("A name must be provided!") + else + print("Hello World, #{@name}!") + end + end +end +``` + +Alternatively, you can make use of the [`#print_error_and_exit`](pathname:///api/0.3/Marten/CLI/Manage/Command/Base.html#print_error_and_exit(msg%2Cexit_code%3D1)-instance-method) method to print a message to the error file descriptor and to exit the execution of the command. + +## Customizing the subcommand name + +By default, management command names are inferred by using their associated class names (eg. a `MyCommand` command class would translate to a `my_command` subcommand). That being said, it should be noted that you can define a custom subcommand name by leveraging the [`#command_name`](pathname:///api/0.3/Marten/CLI/Manage/Command/Base.html#command_name(name%3AString|Symbol)-class-method) class method: + +```crystal +class MyCommand < Marten::CLI::Command + command_name :dummycommand + help "Command that does something" + + def run + # Do something + end +end +``` + +It is also worth mentioning that command aliases can be configured easily by using the [`#command_aliases`](pathname:///api/0.3/Marten/CLI/Manage/Command/Base.html#command_aliases(*aliases%3AString|Symbol)-class-method) helper method. For example: + +```crystal +class MyCommand < Marten::CLI::Command + command_name :test + command_aliases :t + help "Command that does something" + + def run + # Do something + end +end +``` diff --git a/docs/versioned_docs/version-0.3/development/management-commands.md b/docs/versioned_docs/version-0.3/development/management-commands.md new file mode 100644 index 000000000..753abd390 --- /dev/null +++ b/docs/versioned_docs/version-0.3/development/management-commands.md @@ -0,0 +1,74 @@ +--- +title: Management commands +description: Learn the basics of the Marten management CLI tool. +sidebar_label: Management commands +--- + +Marten comes with a built-in command line interface (CLI) that developers can leverage to perform common actions and interact with the framework. This tool provides a set of built-in sub-commands that can be easily extended with new commands. + +## Usage + +The `marten` command is available with each Marten installation, and it is also automatically compiled when the Marten shard is installed. This means that you can either use the `marten` command from anywhere in your system (if the Marten CLI was installed globally like described in [Installation](../getting-started/installation)) or you can run the relative `bin/marten` command from inside your project structure. + +When the `marten` command is executed, it will look for a relative `manage.cr` file to identify your current project, its [settings](./settings), and its installed [applications](./applications), which in turn will define the available sub-commands that you can run. + +The `marten` command is intended to be used as follows: + +```bash +marten [command] [options] [arguments] +``` + +As you can see, the `marten` CLI must be used with a specific **command**, possibly followed by **options** and **arguments** (which may be required or not depending on the considered command). All the built-in commands are listed in the [management commands reference](./reference/management-commands). + +### Displaying help information + +You can display help information about a specific management command by using the `marten` CLI as follows: + +```bash +marten help [command] +marten [command] --help +``` + +### Listing commands + +It is possible to list all the available commands within a project by running the `marten` CLI as follows: + +```bash +marten help +``` + +This should output something like this: + +```bash +Usage: marten [command] [arguments] + +Available commands: + +[marten] + › collectassets + › genmigrations + › listmigrations + › migrate + › new + › resetmigrations + › routes + › serve + › version + +Run a command followed by --help to see command specific information, ex: +marten [command] --help +``` + +All the available commands are listed per application: by default, only `marten` commands are listed obviously (if no other applications are installed), but it should be noted that [applications](./applications) can contribute management commands as well. If that's the case, these additional commands will be automatically listed as well. + +### Shared options + +Each command can accept its own set of arguments and options, but it should be noted that all the available commands always accept the following options: + +* `--error-trace` - Allows showing the full error trace (if a compilation is involved) +* `--no-color` - Disables colored outputs +* `-h, --help` - Displays help information about the considered command + +## Available commands + +Please head over to the [management commands reference](./reference/management-commands) to see a list of all the available management commands. Implementing custom management commands is also a possibility that is documented in [Create custom commands](./how-to/create-custom-commands). diff --git a/docs/versioned_docs/version-0.3/development/reference/management-commands.md b/docs/versioned_docs/version-0.3/development/reference/management-commands.md new file mode 100644 index 000000000..3223cd6b1 --- /dev/null +++ b/docs/versioned_docs/version-0.3/development/reference/management-commands.md @@ -0,0 +1,186 @@ +--- +title: Management commands +description: Management commands reference. +toc_max_heading_level: 2 +--- + +This page provides a reference for all the available management commands and their options. + +## `collectassets` + +**Usage:** `marten collectassets [options]` + +Collects all the assets and copies them into a unique storage. + +Please refer to [Asset handling](../../assets/introduction) to learn more about when and how assets are "collected". + +### Options + +* `--no-input` - Does not show prompts to the user + +### Examples + +```bash +marten collectassets # Collects all the assets +marten collectassets --no-input # Collects all the assets without any prompts +``` + +## `genmigrations` + +**Usage:** `marten genmigrations [options] [app_label]` + +Generates new database migrations. + +This command will scan the table definition corresponding to your current models and will compare it to the equivalent tables that are defined by your migration files. Based on the result of this analysis, a new set of migrations will be created and persisted in your applications' `migrations` folders. Please refer to [Migrations](../../models-and-databases/migrations) to learn more about this mechanism. + +### Options + +* `--empty` - Creates an empty migration + +### Arguments + +* `app_label` - The name of an application to generate migrations for (optional) + +### Examples + +```bash +marten genmigrations # Generates new migrations for all the installed apps +marten genmigrations foo # Generates new migrations for the "foo" app +marten genmigrations foo --empty # Generates an empty migration for the "foo" app +``` + +## `listmigrations` + +**Usage:** `marten listmigrations [options] [app_label]` + +Lists all the available database migrations. + +This command will introspect your project and your installed applications to list the available migrations, and indicate whether they have already been applied or not. Please refer to [Migrations](../../models-and-databases/migrations) to learn more about this mechanism. + +### Options + +* `--db=ALIAS` - Allows specifying the alias of the database on which migrations will be applied or unapplied (default to `default`) + +### Arguments + +* `app_label` - The name of an application to list migrations for (optional) + +### Examples + +```bash +marten listmigrations # Lists all the migrations for all the installed apps +marten listmigrations foo # Lists all the migrations of the "foo" app +``` + +## `migrate` + +**Usage:** `marten migrate [options] [app_label] [migration]` + +Runs database migrations. + +The `migrate` command allows you to apply (or unapply) migrations to your databases. By default, when executed without arguments, it will execute all the non-applied migrations for your installed applications. That being said, it is possible to ensure that only the migrations of a specific application are applied by specifying an additional `app_label` argument. To unapply certain migrations (or to apply some of them up to a certain version only), it is possible to specify another `migration` argument corresponding to the version of a targetted migration. Please refer to [Migrations](../../models-and-databases/migrations) to learn more about this mechanism. + +### Options + +* `--fake` - Allows marking migrations as applied or unapplied without actually running them +* `--plan` - Provides a comprehensive overview of the operations that will be performed by the applied or unapplied migrations +* `--db=ALIAS` - Allows specifying the alias of the database on which migrations will be applied or unapplied (default to `default`) + +### Arguments + +* `app_label` - The name of an application to run migrations for +* `migration` - A migration target (name or version) up to which the DB should be migrated. Use `zero` to unapply all the migrations of a specific application + +### Examples + +```bash +marten migrate # Applies all the non-applied migrations for all the installed apps +marten migrate foo # Applies migrations for the "foo" app +marten migrate foo 202203111821451 # Applies (or unapply) migrations for the "foo" app up until the "202203111821451" migration +``` + +## `new` + +**Usage:** `marten new [options] [type] [name]` + +Initializes a new Marten project or application structure. + +The `new` management command can be used to create either a new project structure or a new [application](../applications) structure. This can be handy when creating new projects or when introducing new applications into an existing project, as it ensures you are following Marten's best practices and conventions. + +The command allows you to fully define the name of your project or application, and in which folder it should be created. + +### Options + +* `-d DIR, --dir=DIR` - An optional destination directory +* `--with-auth` - Adds an authentication application to newly created projects. See [Authentication](../../authentication) to learn more about this capability + +### Arguments + +* `type` - The type of structure to create (must be either `project` or `app`) +* `name` - The name of the project or app to create + +:::tip +The `type` and `name` arguments are optional: if they are not provided, an interactive mode will be used and the command will prompt the user for inputting the structure type, the app or project name, and whether the auth app should be generated. +::: + +### Examples + +```bash +marten new project myblog # Creates a "myblog" project +marten new project myblog --dir=./projects/myblog # Creates a "myblog" project in the "./projects/myblog" folder +marten new app auth # Creates an "auth" application +``` + +## `resetmigrations` + +**Usage:** `marten resetmigrations [options] [app_label]` + +Resets an existing set of migrations into a single one. Please refer to [Resetting migrations](../../models-and-databases/migrations#resetting-migrations) to learn more about this capability. + +### Arguments + +* `app_label` - The name of an application to reset migrations for + +### Examples + +```bash +marten resetmigrations foo # Resets the migrations of the "foo" application +``` + +## `routes` + +**Usage:** `marten routes [options]` + +Displays all the routes of the application. + +## `serve` + +**Usage:** `marten serve [options]` + +Starts a development server that is automatically recompiled when source files change. + +### Options + +* `-b HOST, --bind=HOST` - Allows specifying a custom host to bind +* `-p PORT, --port=PORT` - Allows specifying a custom port to listen for connections + +### Examples + +```bash +marten serve # Starts a development server using the configured host and port +marten serve -p 3000 # Starts a development server by overriding the port +``` + +:::tip +You can also use the alias `s` to start the development server: + +```bash +marten s +``` +::: + +## `version` + +**Usage:** `marten version [options]` + +Shows the Marten version. diff --git a/docs/versioned_docs/version-0.3/development/reference/settings.md b/docs/versioned_docs/version-0.3/development/reference/settings.md new file mode 100644 index 000000000..35be82d64 --- /dev/null +++ b/docs/versioned_docs/version-0.3/development/reference/settings.md @@ -0,0 +1,747 @@ +--- +title: Settings +description: Settings reference. +sidebar_label: Settings +--- + +This page provides a reference for all the available settings that can be used to configure Marten projects. + +## Common settings + +### `allowed_hosts` + +Default: `[] of String` + +An explicit array of allowed hosts for the application. + +The application has to be explicitly configured to serve a list of allowed hosts. This is to mitigate HTTP Host header attacks. The strings in this array can correspond to regular domain names or subdomains (eg. `example.com` or `www.example.com`); when this is the case the Host header of the incoming request will be checked to ensure that it exactly matches one of the configured allowed hosts. + +It is also possible to match all the subdomains of a specific domain by prepending a `.` at the beginning of the host string. For example `.example.com` will matches `example.com`, `www.example.com`, `sub.example.com`, or any other subdomains. Finally, the special `*` string can be used to match any Host value, but this wildcard value should be used with caution as you wouldn't be protected against Host header attacks. + +It should be noted that this setting is automatically set to the following array when a project is running in [debug mode](#debug) (unless it is explicitly set): + +```crystal +[".localhost", "127.0.0.1", "[::1]"] +``` + +### `cache_store` + +Default: `Marten::Cache::Store::Memory.new` + +The global cache store instance. + +This setting allows to configure the cache store returned by the [`Marten#cache`](pathname:///api/0.3/Marten.html#cache%3ACache%3A%3AStore%3A%3ABase-class-method) method (which can be used to perform low-level caching operations), and which is also leveraged for other caching features such as template fragment caching. Please refer to [Caching](../../caching) to learn more about the caching features provided by Marten. + +By default, the global cache store is set to be an in-memory cache (instance of [`Marten::Cache::Store::Memory`](pathname:///api/0.3/Marten/Cache/Store/Memory.html)). In test environments you might want to use the "null store" by assigning an instance of the [`Marten::Cache::Store::Null](pathname:///api/0.3/Marten/Cache/Store/Null.html) to this setting. Additional caching store shards are also maintained under the umbrella of the Marten project or by the community itself and can be used as part of your application depending on your caching requirements. These backends are listed in the [caching stores backend reference](../../caching/reference/stores). + +### `debug` + +Default: `false` + +A boolean allowing to enable or disable debug mode. + +When running in debug mode, Marten will automatically provide detailed information about raised exceptions (including tracebacks) and incoming HTTP requests. As such this mode is mostly useful for development environments. + +### `host` + +Default: `"127.0.0.1"` + +The host the HTTP server running the application will be listening on. + +### `installed_apps` + +Default: `[] of Marten::Apps::Config.class` + +An array of the installed app classes. Each Marten application must define a subclass of [`Marten::Apps::Config`](pathname:///api/0.3/Marten/Apps/Config.html). When those subclasses are specified in the `installed_apps` setting, the applications' models, migrations, assets, and templates will be made available to the considered project. Please refer to [Applications](../applications) to learn more about applications. + +### `log_backend` + +Default: `Log::IOBackend.new(...)` + +The log backend used by the application. Any `Log::Backend` object can be used, which can allow to easily configure how logs are formatted for example. + +### `log_level` + +Default: `Log::Severity::Info` + +The default log level used by the application. Any severity defined in the [`Log::Severity`](https://crystal-lang.org/api/Log/Severity.html) enum can be used. + +### `middleware` + +Default: `[] of Marten::Middleware.class` + +An array of middlewares used by the application. For example: + +```crystal +config.middleware = [ + Marten::Middleware::Session, + Marten::Middleware::I18n, + Marten::Middleware::GZip, +] +``` + +Middlewares are used to "hook" into Marten's request / response lifecycle. They can be used to alter or implement logics based on incoming HTTP requests and the resulting HTTP responses. Please refer to [Middlewares](../../handlers-and-http/middlewares) to learn more about middlewares. + +### `port` + +Default: `8000` + +The port the HTTP server running the application will be listening on. + +### `port_reuse` + +Default: `true` + +A boolean indicating whether multiple processes can bind to the same HTTP server port. + +### `request_max_parameters` + +Default: `1000` + +The maximum number of allowed parameters per request (such as GET or POST parameters). + +A large number of parameters will require more time to process and might be the sign of a denial-of-service attack, which is why this setting can be used. This protection can also be disabled by setting `request_max_parameters` to `nil`. + +### `root_path` + +Default: `nil` + +The root path of the application. + +The root path of the application specifies the actual location of the project sources in your system. This can prove helpful in scenarios where the project was compiled in a specific location different from the final destination where the project sources (and the `lib` folder) are copied. For instance, platforms like Heroku often fall under this category. By configuring the root path, you can ensure that your application correctly locates the required project sources and avoids any discrepancies arising from inconsistent source paths. This can prevent issues related to missing dependencies or missing app-related files (eg. locales, assets, or templates) and make your application more robust and reliable. + +For example, deploying a Marten app on Heroku will usually involves setting the root path as follows: + +```crystal +config.root_path = "/app" +``` + +### `secret_key` + +Default: `""` + +A secret key used for cryptographic signing for the considered Marten project. + +The secret key should be set to a unique and unpredictable string value. The secret key can be used by Marten to encrypt or sign messages (eg. for cookie-based sessions), or by other authentication applications. + +:::warning +The `secret_key` setting value **must** be kept secret. You should never commit this setting value to source control (instead, consider loading it from environment variables for example). +::: + +### `time_zone` + +Default: `Time::Location.load("UTC")` + +The default time zone used by the application when it comes to storing date times in the database and displaying them. Any [`Time::Location`](https://crystal-lang.org/api/Time/Location.html) object can be used. + +### `use_x_forwarded_host` + +Default: `false` + +A boolean indicating whether the `X-Forwarded-Host` header is used to look for the host. This setting can be enabled if the Marten application is served behind a proxy that sets this header. + +### `use_x_forwarded_port` + +Default: `false` + +A boolean indicating if the `X-Forwarded-Port` header is used to determine the port of a request. This setting can be enabled if the Marten application is served behind a proxy that sets this header. + +### `use_x_forwarded_proto` + +Default: `false` + +A boolean indicating if the `X-Forwarded-Proto` header is used to determine whether a request is secure. This setting can be enabled if the Marten application is served behind a proxy that sets this header. For example, if such proxy sets this header to `https`, Marten will assume that the request is secure at the application level **only** if `use_x_forwarded_proto` is set to `true`. + +### `handler400` + +Default: `Marten::Handlers::Defaults::BadRequest` + +The handler class that should generate responses for Bad Request responses (HTTP 400). Please refer to [Error handlers](../../handlers-and-http/error-handlers) to learn more about error handlers. + +### `handler403` + +Default: `Marten::Handlers::Defaults::PermissionDenied` + +The handler class that should generate responses for Permission Denied responses (HTTP 403). Please refer to [Error handlers](../../handlers-and-http/error-handlers) to learn more about error handlers. + +### `handler404` + +Default: `Marten::Handlers::Defaults::PageNotFound` + +The handler class that should generate responses for Not Found responses (HTTP 404). Please refer to [Error handlers](../../handlers-and-http/error-handlers) to learn more about error handlers. + +### `handler500` + +Default: `Marten::Handlers::Defaults::ServerError` + +The handler class that should generate responses for Internal Error responses (HTTP 500). Please refer to [Error handlers](../../handlers-and-http/error-handlers) to learn more about error handlers. + +### `x_frame_options` + +Default: `"DENY"` + +The value to use for the X-Frame-Options header when the associated middleware is used. The value of this setting will be used by the [`Marten::Middleware::XFrameOptions`](../../handlers-and-http/reference/middlewares#x-frame-options-middleware) middleware when inserting the X-Frame-Options header in HTTP responses. + +## Assets settings + +Assets settings allow configuring how Marten should interact with [assets](../../assets/introduction). These settings are all available under the `assets` namespace: + +```crystal +config.assets.root = "assets" +config.assets.url = "/assets/" +``` + +### `app_dirs` + +Default: `true` + +A boolean indicating whether assets should be looked for inside installed application folders. When this setting is set to `true`, this means that assets provided by installed applications will be collected by the `collectassets` command (please refer to [Asset handling](../../assets/introduction) for more details regarding how to manage assets in your project). + +### `dirs` + +Default: `[] of String` + +An array of directories where assets should be looked for. The order of these directories is important as it defines the order in which assets are searched for. + +It should be noted that path objects or symbols can also be used to configure this setting: + +```crystal +config.assets.dirs = [ + Path["src/path1/assets"], + :"src/path2/assets", +] +``` + +### `manifests` + +Default: `[] of String` + +An array of paths to manifest JSON files to use to resolve assets URLs. Manifest files will be used to return the right fingerprinted asset path for a generic path, which can be useful if your asset bundling strategy support this. + +### `root` + +Default: `"assets"` + +A string containing the absolute path where collected assets will be persisted (when running the `collectassets` command). By default, assets will be persisted in a folder that is relative to the Marten project's directory. Obviously, this folder should be empty before running the `collectassets` command in order to not overwrite existing files: assets should be defined as part of your applications' `assets` folders instead. + +:::info +This setting is only used if `assets.storage` is `nil`. +::: + +### `storage` + +Default: `nil` + +An optional storage object, which must be an instance of a subclass of [`Marten::Core::Store::Base`](pathname:///api/0.3/Marten/Core/Storage/Base.html). This storage object will be used when collecting asset files to persist them in a given location. + +By default this setting value is set to `nil`, which means that a [`Marten::Core::Store::FileSystem`](pathname:///api/0.3/Marten/Core/Storage/FileSystem.html) storage is automatically constructed by using the `assets.root` and `assets.url` setting values: in this situation, asset files are collected and persisted in a local directory, and it is expected that they will be served from this directory by the web server running the application. + +A specific storage can be set instead to ensure that collected assets are persisted somewhere else in the cloud and served from there (for example in an Amazon's S3 bucket). When this is the case, the `assets.root` and `assets.url` setting values are basically ignored and are overridden by the use of the specified storage. + +### `url` + +Default: `"/assets/"` + +The base URL to use when exposing asset URLs. This base URL will be used by the default [`Marten::Core::Store::FileSystem`](pathname:///api/0.3/Marten/Core/Storage/FileSystem.html) storage to construct asset URLs. For example, requesting a `css/App.css` asset might generate a `/assets/css/App.css` URL by default. + +:::info +This setting is only used if `assets.storage` is `nil`. +::: + +## CSRF settings + +CSRF settings allow configuring how Cross-Site Request Forgeries (CSRF) attack protection measures are implemented within the considered Marten project. Please refer to [Cross-Site Request Forgery protection](../../security/csrf) for more details about this topic. + +The following settings are all available under the `csrf` namespace: + +```crystal +config.csrf.protection_enabled = true +config.csrf.cookie_name = "csrf-token" +``` + +### `cookie_domain` + +Default: `nil` + +An optional domain to use when setting the CSRF cookie. This can be used to share the CSRF cookie across multiple subdomains for example. For example, setting this option to `.example.com` will make it possible to send a POST request from a form on one subdomain (eg. `foo.example.com`) to another subdomain (eg. `bar.example.com `). + +### `cookie_http_only` + +Default: `false` + +A boolean indicating whether client-side scripts should be prevented from accessing the CSRF token cookie. If this option is set to `true`, Javascript scripts won't be able to access the CSRF cookie. + +### `cookie_max_age` + +Default: `31_556_952` (approximately one year) + +The max age (in seconds) of the CSRF cookie. + +### `cookie_name` + +Default: `"csrftoken"` + +The name of the cookie to use for the CSRF token. This cookie name should be different than any other cookies created by your application. + +### `cookie_same_site` + +Default: `"Lax"` + +The value of the [SameSite flag](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie/SameSite) to use for the CSRF cookie. Accepted values are `"Lax"`, `"Strict"`, or `"None"`. + +### `cookie_secure` + +Default: `false` + +A boolean indicating whether to use a secure cookie for the CSRF cookie. Setting this to `true` will force browsers to send the cookie with an encrypted request over the HTTPS protocol only. + +### `protection_enabled` + +Default: `true` + +A boolean indicating if the CSRF protection is enabled globally. When set to `true`, handlers will automatically perform a CSRF check to protect unsafe requests (ie. requests whose methods are not `GET`, `HEAD`, `OPTIONS`, or `TRACE`). Regardless of the value of this setting, it is always possible to explicitly enable or disable CSRF protection on a per-handler basis. See [Cross-Site Request Forgery protection](../../security/csrf) for more details. + +### `trusted_origins` + +Default: `[] of String` + +An array of trusted origins. + +These origins will be trusted for CSRF-protected requests (such as POST requests) and they will be used to check either the `Origin` or the `Referer` header depending on the request scheme. This is done to ensure that a specific subdomain such as `sub1.example.com` cannot issue a POST request to `sub2.example.com`. To enable CSRF-protected requests over different origins, it's possible to add trusted origins to this array. For example `https://sub1.example.com` can be configured as a trusted domain that way, but it's possible to allow CSRF-protected requests for all the subdomains of a specific domain by using `https://*.example.com`. + +For example: + +```crystal +config.csrf.trusted_origins = [ + "https://*.example.com", + "https://other.example.org", +] +``` + +## Content-Security-Policy settings + +These settings allow configuring how the [`Marten::Middleware::ContentSecurityPolicy`](../../handlers-and-http/reference/middlewares#content-security-policy-middleware) middleware behaves and the actual directives of the Content-Security-Policy header that are set by this middleware. + +```crystal +config.content_security_policy.report_only = true +config.content_security_policy.default_policy.default_src = [:self, "other"] +``` + +Please refer to [Content Security Policy](../../security/content-security-policy) to learn more about the Content-Security-Policy header protection. + +:::tip +[Content-Security-Policy](https://www.w3.org/TR/CSP/) is a complicated header and there are possibly many values you may need to tweak. Make sure you understand it before configuring the below settings. +::: + +### `default_policy` + +Default: `Marten::HTTP::ContentSecurityPolicy.new` + +The default Content-Security-Policy object. + +This [`Marten::HTTP::ContentSecurityPolicy`](pathname:///api/0.3/Marten/HTTP/ContentSecurityPolicy.html) object will be used to set the Content-Security-Policy header when the [`Marten::Middleware::ContentSecurityPolicy`](../../handlers-and-http/reference/middlewares#content-security-policy-middleware) middleware is used. + +All the attributes that can be set on this [`Marten::HTTP::ContentSecurityPolicy`](pathname:///api/0.3/Marten/HTTP/ContentSecurityPolicy.html) object through the use of methods such as [`#default_src=`](pathname:///api/0.3/Marten/HTTP/ContentSecurityPolicy.html#default_src%3D(value%3AArray|Nil|String|Symbol|Tuple)-instance-method) or [`#frame_src=`](pathname:///api/0.3/Marten/HTTP/ContentSecurityPolicy.html#frame_src%3D(value%3AArray|Nil|String|Symbol|Tuple)-instance-method) can also be used directly on the `content_security_policy` setting object. For example: + +```crystal +config.content_security_policy.default_src = [:self, "other"] +config.content_security_policy.block_all_mixed_content = true +``` + +### `nonce_directives` + +Default: `["script-src", "style-src"]` + +An array of directives where a dynamically-generated nonce will be included. + +For example, if this setting is set to `["script-src"]`, a `nonce-` value will be added to the `script-src` directive in the Content-Security-Policy header value. + +### `report_only` + +Default: `false` + +A boolean indicating whether policy violations are reported without enforcing them. + +If this setting is set to `true`, the [`Marten::Middleware::ContentSecurityPolicy`](../../handlers-and-http/reference/middlewares#content-security-policy-middleware) middleware will set a [Conten-Security-Policy-Report-Only](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy-Report-Only) header instead of the regular Content-Security-Policy header. Doing so can be useful to experiment with policies without enforcing them. + +## Database settings + +These settings allow configuring the databases used by the considered Marten project. At least one default database must be configured if your project makes use of [models](../../models-and-databases/introduction), and additional databases can optionally be configured as well. + +```crystal +# Default database +config.database do |db| + db.backend = :sqlite + db.name = "default_db.db" +end + +# Additional database +config.database :other do |db| + db.backend = :sqlite + db.name = "other_db.db" +db +``` + +Configuring other database backends such as MySQL or PostgreSQL usually involves specifying more connection parameters (eg. user, password, etc). For example: + +```crystal +config.database do |db| + db.backend = :postgresql + db.host = "localhost" + db.name = "my_db" + db.user = "my_user" + db.password = "my_passport" +end +``` + +The following options are all available when configuring a database configuration object, which is available by opening a block with the `#database` method (like in the above examples). + +### `backend` + +Default: `nil` + +The database backend to use for connecting to the considered database. Marten supports three backends presently: + +* `:mysql` +* `:postgresql` +* `:sqlite` + +### `checkout_timeout` + +Default: `5.0` + +The number of seconds to wait for a connection to become available when the max pool size is reached. + +### `host` + +Default: `nil` + +A string containing the host used to connect to the database. No value means that the host will be localhost. + +### `initial_pool_size` + +Default: `1` + +The initial number of connections created for the database connections pool. + +### `max_idle_pool_size` + +Default: `1` + +The maximum number of idle connections for the database connections pool. Concretely, this means that when released, a connection will be closed only if there are already `max_idle_pool_size` idle connections. + +### `max_pool_size` + +Default: `0` + +The maximum number of connections that will be held by the database connections pool. When set to `0`, this means that there is no limit to the number of connections. + +### `name` + +Default: `nil` + +The name of the database to connect to. If you use the `sqlite` backend, this can be a string or a `Path` object containing the path (absolute or relative) to the considered database path. + +### `options` + +Default: `{} of String => String` + +A set of additional database options. This setting can be used to set additional database options that may be required in order to connect to the database at hand. + +For example: + +```crystal +config.database do |db| + db.backend = :postgresql + db.host = "localhost" + db.name = "my_db" + db.user = "my_user" + db.password = "my_passport" + // highlight-next-line + db.options = {"sslmode" => "disable"} +end +``` + +### `password` + +Default: `nil` + +A string containing the password to use to connect to the configured database. + +### `port` + +Default: `nil` + +The port to use to connect to the configured database. No value means that the default port will be used. + +### `retry_attempts` + +Default: `1` + +The maximum number of attempts to retry re-establishing a lost connection. + +### `retry_delay` + +Default: `1.0` + +The delay to wait between each retry at re-establishing a lost connection. + +### `user` + +Default: `nil` + +A string containing the name of the user that should be used to connect to the configured database. + +## Emailing settings + +Emailing settings allow configuring emailing-related settings. Please refer to [Emailing](../../emailing) for more details about how to define and send emails in your projects. + +The following settings are all available under the `emailing` namespace: + +```crystal +config.emailing.from_address = "no-reply@example.com" +config.emailing.backend = Marten::Emailing::Backend::Development.new(print_emails: true) +``` + +### `backend` + +Default: `Marten::Emailing::Backend::Development.new` + +The backend to use when it comes to send emails. Emailing backends define _how_ emails are actually sent. + +By default, a development backend (instance of [`Marten::Emailing::Backend::Dev`](pathname:///api/0.3/Marten/Emailing/Backend/Development.html)) is used: this backend "collects" all the emails that are "delivered" by default (which can be used in specs in order to test sent emails), but it can also be configured to print email details to the standard output if necessary (see the [emailing backend reference](../../emailing/reference/backends) for more details about this capability). + +Additional emailing backend shards are also maintained under the umbrella of the Marten project or by the community itself and can be used as part of your application depending on your specific email sending requirements. These backends are listed in the [emailing backend reference](../../emailing/reference/backends#other-backends). + +### `from_address` + +Default: `"webmaster@localhost"` + +The default from address used in emails. Email definitions that don't specify a "from" address explicitly will use this email address automatically for the sender email. It should be noted that this from email address can be defined as a string or as a [`Marten::Emailing::Address`](pathname:///api/0.3/Marten/Emailing/Address.html) object (which allows to specify the name AND the address of the sender email). + +## I18n settings + +I18n settings allow configuring internationalization-related settings. Please refer to [Internationalization](../../i18n) for more details about how to leverage translations and localized content in your projects. + +:::info +Marten makes use of [crystal-i18n](https://crystal-i18n.github.io/) to handle translations and locales. Further [configuration options](https://crystal-i18n.github.io/configuration.html) are also provided by this shard and can be leveraged by any Marten projects if necessary. +::: + +The following settings are all available under the `i18n` namespace: + +```crystal +config.i18n.default_locale = :fr +``` + +### `available_locales` + +Default: `nil` + +Allows defining the locales that can be activated to perform translation lookups and localizations. For example: + +```crystal +config.i18n.available_locales = [:en, :fr] +``` + +### `default_locale` + +Default: `"en"` + +The default locale used by the Marten project. + +### `locale_cookie_name` + +Default: `"marten_locale"` + +The name of the cookie to use for saving the locale of the current user and activating the right locale (when the [`Marten::Middleware::I18n`](../../handlers-and-http/reference/middlewares#i18n-middleware) middleware is used). See [Internationalization](../../i18n/introduction) to learn more about this capability. + +## Media files settings + +Media files settings allow configuring how Marten should interact with [media files](../../files/managing-files). These settings are all available under the `media_files` namespace: + +```crystal +config.media_files.root = "files" +config.media_files.url = "/files/" +``` + +### `root` + +Default: `"media"` + +A string containing the absolute path where uploaded files will be persisted. By default uploaded files will be persisted in a folder that is relative to the Marten project's directory. + +:::info +This setting is only used if `media_files.storage` is `nil`. +::: + +### `storage` + +Default: `nil` + +An optional storage object, which must be an instance of a subclass of [`Marten::Core::Store::Base`](pathname:///api/0.3/Marten/Core/Storage/Base.html). This storage object will be used when uploading files to persist them in a given location. + +By default, this setting value is set to `nil`, which means that a [`Marten::Core::Store::FileSystem`](pathname:///api/0.3/Marten/Core/Storage/FileSystem.html) storage is automatically constructed by using the `media_files.root` and `media_files.url` setting values: in this situation, media files are persisted in a local directory, and it is expected that they will be served from this directory by the web server running the application. + +A specific storage can be set instead to ensure that uploaded files are persisted somewhere else in the cloud and served from there (for example in an Amazon's S3 bucket). When this is the case, the `media_files.root` and `media_files.url` setting values are basically ignored and are overridden by the use of the specified storage. + +### `url` + +Default: `"/media/"` + +The base URL to use when exposing media files URLs. This base URL will be used by the default [`Marten::Core::Store::FileSystem`](pathname:///api/0.3/Marten/Core/Storage/FileSystem.html) storage to construct media files URLs. For example, requesting a `foo/bar.txt` file might generate a `/media/foo/bar.txt` URL by default. + +:::info +This setting is only used if `media_files.storage` is `nil`. +::: + +## Sessions settings + +Sessions settings allow configuring how Marten should handle [sessions](../../handlers-and-http/introduction#using-sessions). These settings are all available under the `sessions` namespace: + +```crystal +config.sessions.cookie_name = "_sessions" +config.sessions.store = :cookie +``` + +### `cookie_domain` + +Default: `nil` + +An optional domain to use when setting the session cookie. This can be used to share the session cookie across multiple subdomains. + +### `cookie_http_only` + +Default: `false` + +A boolean indicating whether client-side scripts should be prevented from accessing the session cookie. If this option is set to `true`, Javascript scripts won't be able to access the session cookie. + +### `cookie_max_age` + +Default: `1_209_600` (two weeks) + +The max age (in seconds) of the session cookie. + + +### `cookie_name` + +Default: `"sessionid"` + +The name of the cookie to use for the session token. This cookie name should be different than any other cookies created by your application. + +### `cookie_same_site` + +Default: `"Lax"` + +The value of the [SameSite flag](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie/SameSite) to use for the session cookie. Accepted values are `"Lax"`, `"Strict"`, or `"None"`. + +### `cookie_secure` + +Default: `false` + +A boolean indicating whether to use a secure cookie for the session cookie. Setting this to `true` will force browsers to send the cookie with an encrypted request over the HTTPS protocol only. + +### `store` + +Default: `"cookie"` + +A string containing the identifier of the store used to handle sessions. + +By default, sessions are stored within a single cookie. Cookies have a 4K size limit, which is usually sufficient to persist things like a user ID and flash messages. Other stores can be implemented and leveraged to store sessions data; see [Sessions](../../handlers-and-http/sessions) for more details about this capability. + +## SSL redirect settings + +SSL redirect settings allow to configure how Marten should redirect non-HTTPS requests to HTTPS when the [`Marten::Middleware::SSLRedirect`](../../handlers-and-http/reference/middlewares#ssl-redirect-middleware) middleware is used: + +```crystal +config.ssl_redirect.host = "example-redirect.com" +config.exempted_paths = [/^\/no-ssl\/$/] +``` + +### `exempted_paths` + +Default: `[] of Regex | String` + +Allows to set the array of paths that should be exempted from HTTPS redirects. Both strings and regexes are accepted. + +### `host` + +Default: `nil` + +Allows to set the host that should be used when redirecting non-HTTPS requests. If set to `nil`, the HTTPS redirect will be performed using the request's host. + +## Strict transport security policy settings + +Strict transport security policy settings allow to configure how Marten should set the HTTP Strict-Transport-Security response header when the [`Marten::Middleware::StrictTransportSecurity`](../../handlers-and-http/reference/middlewares#strict-transport-security-middleware) middleware is used: + +```crystal +config.strict_transport_security.max_age = 3_600 +config.strict_transport_security.include_sub_domains = true +``` + +### `include_sub_domains` + +Default: `false` + +Defines whether the `includeSubDomains` directive should be inserted into the HTTP Strict-Transport-Security response header. When this directive is set, this means that the policy will also apply to all the site's subdomains. + +:::caution +You should be careful when enabling this option as this will prevent browsers from connecting to your site's subdomains using HTTP for the duration defined by the [`max_age`](#max_age) setting. +::: + +### `max_age` + +Default: `nil` + +Defines the duration in seconds that browsers should remember that the web app must be accessed using HTTPS only. A `nil` value means that the HTTP Strict-Transport-Security response header is not inserted in responses (which is equivalent to not using the [`Marten::Middleware::StrictTransportSecurity`](../../handlers-and-http/reference/middlewares#strict-transport-security-middleware) middleware). + +:::caution +You should be careful when defining a value for this setting because this will prevent browsers from connecting to your site using HTTP for the duration you specified. +::: + +### `preload` + +Default: `false` + +Defines whether the `preload` directive should be inserted into the HTTP Strict-Transport-Security response header. Setting this to `true` means that you allow your site to be submitted to the [HSTS browser preload list](https://hstspreload.org/) by browsers. + +## Templates settings + +Templates settings allow configuring how Marten discovers and renders [templates](../../templates). These settings are all available under the `templates` namespace: + +```crystal +config.templates.app_dirs = false +config.templates.cached = false +``` + +### `app_dirs` + +Default: `true` + +A boolean indicating whether templates should be looked for inside installed application folders (local `templates` directories). When this setting is set to `true`, this means that templates provided by installed applications can be loaded and rendered by the templates engine. Otherwise, it would not be possible to load and render these application templates. + +### `cached` + +Default: `false` + +A boolean indicating whether templates should be kept in a memory cache upon being loaded and parsed. This setting should likely be set to `false` in development environments (where changes to templates are frequent) and set to `true` in production environments (to avoid loading and parsing the same templates multiple times). + +### `context_producers` + +Default: `[] of Marten::Template::ContextProducer.class` + +An array of context producer classes. Context producers are helpers that ensure that common variables are automatically inserted in the template context whenever a template is rendered. See [Using context producers](../../templates/introduction#using-context-producers) to learn more about this capability. + +### `dirs` + +Default: `[] of String` + +An array of directories where templates should be looked for. The order of these directories is important as it defines the order in which templates are searched for when requesting a template for a given path (eg. `foo/bar/template.html`). + +It should be noted that path objects or symbols can also be used to configure this setting: + +```crystal +config.templates.dirs = [ + Path["src/path1/templates"], + :"src/path2/templates", +] +``` diff --git a/docs/versioned_docs/version-0.3/development/settings.md b/docs/versioned_docs/version-0.3/development/settings.md new file mode 100644 index 000000000..3198df397 --- /dev/null +++ b/docs/versioned_docs/version-0.3/development/settings.md @@ -0,0 +1,72 @@ +--- +title: Settings +description: Learn the basics of Marten settings. +sidebar_label: Settings +--- + +Marten projects can be configured through the use of settings files. This section explains how settings work, how they relate to environments, and how they can be altered. + +## Overview + +Settings will be usually defined under a `config/settings` folder at the root of your project structure. There are no strict requirements regarding _where_ settings are defined nor how they are organized, but as a general rule of thumb, it is recommended to organize settings on a per-environment basis (shared settings, development settings, production settings, etc). + +In such configuration, you will usually define shared settings (settings that are shared across all your environments) in a dedicated settings file (eg. `config/settings/base.cr`) and other environment-specific settings in other files (eg. `config/settings/development.cr`). + +To define settings, it is necessary to access the global Marten configuration object through the use of the [`Marten#configure`](pathname:///api/0.3/Marten.html#configure(env%3ANil|String|Symbol%3Dnil%2C%26)-class-method) method. This method returns a [`Marten::Conf::GlobalSettings`](pathname:///api/0.3/Marten/Conf/GlobalSettings.html) object that you can use to define setting values. For example: + +```crystal +Marten.configure do |config| + config.installed_apps = [ + FooApp, + BarApp, + ] + + config.middleware = [ + Marten::Middleware::Session, + Marten::Middleware::Flash, + Marten::Middleware::GZip, + Marten::Middleware::XFrameOptions, + ] + + config.database do |db| + db.backend = :postgresql + db.name = "dummypress" + db.host = "localhost" + db.password = "" + end +end +``` + +It should be noted that the [`Marten#configure`](pathname:///api/0.3/Marten.html#configure(env%3ANil|String|Symbol%3Dnil%2C%26)-class-method) method can be called with an additional argument to ensure that the underlying settings are defined for a specific environment only: + +```crystal +Marten.configure :development do |config| + config.secret_key = "INSECURE" +end +``` + +:::caution +You should avoid altering setting values outside of the configuration block provided by the [`Marten#configure`](pathname:///api/0.3/Marten.html#configure(env%3ANil|String|Symbol%3Dnil%2C%26)-class-method) method. Most settings are "read" and applied when the Marten project is set up, that is before the server actually starts. Changing these setting values afterward won't produce any meaningful result. +::: + +## Environments + +When creating new projects by using the [`new`](./reference/management-commands#new) management command, the following environments will be created automatically: + +* Development (settings defined in `config/settings/development.cr`) +* Test (settings defined in `config/settings/test.cr`) +* Production (settings defined in `config/settings/production.cr`) + +When your application is running, Marten will rely on the `MARTEN_ENV` environment variable to determine the current environment. If this environment variable is not found, the environment will automatically default to `development`. The value you specify in the `MARTEN_ENV` environment variable must correspond to the argument you pass to the [`Marten#configure`](pathname:///api/0.3/Marten.html#configure(env%3ANil|String|Symbol%3Dnil%2C%26)-class-method) method. + +It should be noted that the current environment can be retrieved through the use of the [`Marten#env`](pathname:///api/0.3/Marten.html#env-class-method) method, which returns a [`Marten::Conf::Env`](pathname:///api/0.3/Marten/Conf/Env.html) object. For example: + +```crystal +Marten.env # => +Marten.env.id # => "development" +Marten.env.development? # => true +``` + +## Available settings + +All the available settings are listed in the [settings reference](./reference/settings). diff --git a/docs/versioned_docs/version-0.3/development/testing.md b/docs/versioned_docs/version-0.3/development/testing.md new file mode 100644 index 000000000..44b46d6fd --- /dev/null +++ b/docs/versioned_docs/version-0.3/development/testing.md @@ -0,0 +1,253 @@ +--- +title: Testing +description: Learn how to test your Marten project. +sidebar_label: Testing +--- + +This section covers the basics regarding how to test a Marten project and the various tools that you can leverage in this regard. + +## The basics + +You should test your Marten project to ensure that it adheres to the specifications it was built for. Like any Crystal project, Marten lets you write "specs" (see the [official documentation related to testing in Crystal](https://crystal-lang.org/reference/guides/testing.html) to learn more about those). + +By default, when creating a project through the use of the [`new`](./reference/management-commands#new) management command, Marten will automatically create a `spec/` folder at the root of your project structure. This folder contains a unique `spec_helper.cr` file allowing you to initialize the test environment for your Marten project. + +This file should look something like this: + +```crystal title=spec/spec_helper.cr +ENV["MARTEN_ENV"] = "test" + +require "spec" +require "marten" +require "marten/spec" + +require "../src/project" +``` + +As you can see, the `spec_helper.cr` file forces the Marten environment variable to be set to `test` and requires the spec library as well as Marten and your actual project. This file should be required by all your spec files. + +:::info +It's very important to require `marten/spec` in your top-level spec helper as this will ensure that the mandatory spec callbacks are configured for your spec suite (eg. in order to ensure that your database is properly set up before each spec is executed). +::: + +When it comes to running your tests, you can simply make use of the standard [`crystal spec`](https://crystal-lang.org/reference/man/crystal/index.html#crystal-spec) command. + +## Writing tests + +To write tests, you should write regular [specs](https://crystal-lang.org/reference/guides/testing.html) and ensure that your spec files always require the `spec/spec_helper.cr` file. + +For example: + +```crystal +require "./spec_helper" + +describe MySuperAbstraction do + describe "#foo" do + it "returns bar" do + obj = MySuperAbstraction.new + obj.foo.should eq "bar" + end + end +end +``` + +You are encouraged to organize your spec files by following the structure of your projects. For example, you could create a `models` folder and define specs related to your models in it. + +:::tip +When organizing spec files across multiple folders, one good practice is to define a `spec_helper.cr` file at each level of your folders structure. These additional `spec_helper.cr` files should require the same file from the parent folder. + +For example: + +```crystal title=spec/models/spec_helper.cr +require "../spec_helper" +``` + +```crystal title=spec/models/article_spec.cr +require "./spec_helper" + +describe Article do + # ... +end +``` +::: + +## Running tests + +As mentioned before, running specs involves making use of the standard [`crystal spec`](https://crystal-lang.org/reference/man/crystal/index.html#crystal-spec) command. + +### The test environment + +By default, the [`new`](./reference/management-commands#new) management command always creates a `test` environment when generating new projects. As such, you should ensure that the `MARTEN_ENV` environment variable is set to `test` when running your Crystal specs. It should also be reminded that this `test` environment is associated with a dedicated settings file where test-related settings can be specified and/or overridden if necessary (see [Settings](./settings#environments) for more details about this). + +### The test database + +Marten **must** use a different database when running tests in order to not tamper with your regular database. Indeed, the database used in the context of specs will be flushed and generated automatically every time the specs suite is executed. You should not set these database names to the same names as the ones used for your development or production environments. If test database names are not explicitly set, your specs suite won't be allowed to run at all. + +One way to ensure you use a dedicated database specifically for tests is to override the [`database`](./reference/settings#database-settings) settings as follows: + +```crystal title=config/settings/test.cr +Marten.configure :test do |config| + config.database do |db| + db.name = "my_project_test" + end +end +``` + +## Testing tools + +Marten provides some tools that can become useful when writing specs. + +### Using the test client + +The test client is an abstraction that is provided when requiring `marten/spec` and that acts as a very basic web client. This tool allows you to easily test your handlers and the various routes of your application by issuing requests and by introspecting the returned responses. + +By leveraging the test client, you can easily simulate various requests (eg. GET or POST requests) for specific URLs and observe the returned responses. While doing so, you can introspect the response properties (such as its status code, content, and headers) in order to verify that your handlers behave as expected. + +#### A simple example + +To use the test client, you can either initialize a [`Marten::Spec::Client`](pathname:///api/0.3/Marten/Spec/Client.html) object or make use of the per-spec test client that is provided by the [`Marten::Spec#client`](pathname:///api/0.3/Marten/Spec.html#client%3AClient-class-method) method. Initializing new [`Marten::Spec::Client`](pathname:///api/0.3/Marten/Spec.html#client%3AClient-class-method) objects allow you to set client-wide properties, like a default content type. + +:::info +Note that the client returned by the [`Marten::Spec#client`](pathname:///api/0.3/Marten/Spec.html#client%3AClient-class-method) method is memoized and is reset after _each_ spec execution. +::: + +Let's have a look at a simple way to use the test client and verify the corresponding responses: + +```crystal +describe MyRedirectHandler do + describe "#get" do + it "returns the expected redirect response" do + response = Marten::Spec.client.get("/my-redirect-handler", query_params: {"foo" => "bar"}) + + response.status.should eq 302 + response.headers["Location"].should eq "/redirected" + end + end +end +``` + +:::tip +In the above example we are simply specifying a "raw" path by hardcoding its value. In a real scenario, you will likely want to [resolve your handler URLs](../handlers-and-http/routing#reverse-url-resolutions) using the [`Marten::Routing::Map#reverse`](pathname:///api/0.3/Marten/Routing/Map.html#reverse(name%3AString|Symbol%2Cparams%3AHash(String|Symbol%2CParameter%3A%3ATypes))-instance-method) method of the main routes map (that way, you don't hardcode route paths in your specs). For example + +```crystal +url = Marten.routes.reverse("article_detail", pk: 42) +response = Marten::Spec.client.get(url, query_params: {"foo" => "bar"}) +``` +::: + +Here we are simply issuing a GET request (by leveraging the [`#get`](pathname:///api/0.3/Marten/Spec/Client.html#get(path%3AString%2Cquery_params%3AHash|NamedTuple|Nil%3Dnil%2Ccontent_type%3AString|Nil%3Dnil%2Cheaders%3AHash|NamedTuple|Nil%3Dnil%2Csecure%3Dfalse)%3AMarten%3A%3AHTTP%3A%3AResponse-instance-method) test client method) and testing the obtained response. A few things can be noted: + +* The test client does not require your project's server to be running: internally it uses a lightweight server handlers chain that ensures that your project's middlewares are applied and that the URL you requested is resolved and mapped to the right handler +* Only the path to the handler needs to be specified when issuing requests (eg. `/foo/bar`) + +Note that you can also issue other types of requests by leveraging methods like [`#post`](pathname:///api/0.3/Marten/Spec/Client.html#post(path%3AString%2Cdata%3AHash|NamedTuple|Nil|String%3Dnil%2Cquery_params%3AHash|NamedTuple|Nil%3Dnil%2Ccontent_type%3AString|Nil%3Dnil%2Cheaders%3AHash|NamedTuple|Nil%3Dnil%2Csecure%3Dfalse)%3AMarten%3A%3AHTTP%3A%3AResponse-instance-method), [`#put`](pathname:///api/0.3/Marten/Spec/Client.html#put(path%3AString%2Cdata%3AHash|NamedTuple|Nil|String%3Dnil%2Cquery_params%3AHash|NamedTuple|Nil%3Dnil%2Ccontent_type%3AString|Nil%3Dnil%2Cheaders%3AHash|NamedTuple|Nil%3Dnil%2Csecure%3Dfalse)%3AMarten%3A%3AHTTP%3A%3AResponse-instance-method), or [`#delete`](pathname:///api/0.3/Marten/Spec/Client.html#delete(path%3AString%2Cdata%3AHash|NamedTuple|Nil|String%3Dnil%2Cquery_params%3AHash|NamedTuple|Nil%3Dnil%2Ccontent_type%3AString|Nil%3Dnil%2Cheaders%3AHash|NamedTuple|Nil%3Dnil%2Csecure%3Dfalse)%3AMarten%3A%3AHTTP%3A%3AResponse-instance-method). For example: + +```crystal +describe MySchemaHandler do + describe "#post" do + it "validates the data and redirects" do + response = Marten::Spec.client.post("/my-schema-handler", data: {"first_name" => "John", "last_name" => "Doe"}) + + response.status.should eq 302 + response.headers["Location"].should eq "/redirected" + end + end +end +``` + +:::info +By default, CSRF checks are disabled for requests issued by the test client. If for some reasons you need to ensure that those are enabled, you can initialize a [`Marten::Spec::Client`](pathname:///api/0.3/Marten/Spec/Client.html) object with `disable_request_forgery_protection: false`. +::: + +#### Introspecting responses + +Responses returned by the test client are instances of the standard [`Marten::HTTP::Response`](pathname:///api/0.3/Marten/HTTP/Response.html) class. As such you can easily access response attributes such as the status code, the content and content type, cookies, and headers in your specs in order to verify that the expected response was returned by your handler. + +#### Exceptions + +It is important to note that exceptions raised in your handlers will be visible from your spec. This means that you should use the standard [`#expect_raises`](https://crystal-lang.org/api/Spec/Expectations.html#expect_raises%28klass%3AT.class%2Cmessage%3AString%7CRegex%7CNil%3Dnil%2Cfile%3D__FILE__%2Cline%3D__LINE__%2C%26%29forallT-instance-method) expectation helper to verify that these exceptions are indeed raised. + +#### Session and cookies + +Test clients are always stateful: if a handler sets a cookie in the returned response, then this cookie will be stored in the client's cookie store (available via the [`#cookies`](pathname:///api/0.3/Marten/Spec/Client.html#cookies-instance-method) method) and will be automatically sent for subsequent requests issued by the client. + +The same goes for session values: such values can be set using the session store returned by the [`#sessions`](pathname:///api/0.3/Marten/Spec/Client.html#session-instance-method) client method. If you set session values in this store prior to any request, the matched handler will have access to them and the new values that are set by the handler will be available for further inspection once the response is returned. These session values are also maintained between requests issued by a single client. + +For example: + +```crystal +describe MyHandler do + describe "#get" do + it "renders the expected content if the right value is in the session" do + Marten::Spec.client.session["foo"] = "bar" + + url = Marten.routes.reverse("initiate_request") + response = Marten::Spec.client.get(url) + + response.status.should eq 200 + response.content.includes?("Initiate request").should be_true + end + end +end +``` + +#### Testing client and authentication + +When using the [marten-auth](https://github.com/martenframework/marten-auth) shard and the built-in [authentication](../authentication), a few additional helpers can be leveraged in order to easily sign in/sign out users while using the test client: + +* The `#sign_in` method can be used to simulate the effect of a signed-in user. This means that the user ID will be persisted into the test client session and that requests issued with it will be associated with the considered user +* The `#sign_out` method can be used to ensure that any signed-in user is logged out and that the session is flushed + +For example: + +```crystal +describe MyHandler do + describe "#get" do + it "shows the profile page of the authenticated user" do + user = Auth::User.create!(email: "test@example.com") do |user + user.set_password("insecure") + end + + url = Marten.routes.reverse("auth:profile") + + Marten::Spec.client.sign_in(user) + response = Marten::Spec.client.get(url) + + response.status.should eq 200 + response.content.includes?("Profile").should be_true + end + end +end +``` + +### Collecting emails + +If your code is sending [emails](../emailing/introduction), you might want to test that these emails are sent as expected. To do that, you can leverage the [development emailing backend](../emailing/reference/backends#development-backend) to ensure that sent emails are collected as part of each spec execution. + +To do that, the emailing backend needs to be initialized with `collect_emails: true` when configuring the [`emailing.backend`](./reference/settings#backend-1) setting. For example: + +```crystal title=config/settings/test.cr +Marten.configure :test do |config| + config.backend = Marten::Emailing::Backend::Development.new(collect_emails: true) +end +``` + +Doing so will ensure that all sent emails are "collected" for further inspection. You can easily retrieve collected emails by calling the [`Marten::Spec#delivered_emails`](pathname:///api/0.3/Marten/Spec.html#delivered_emails%3AArray(Emailing%3A%3AEmail)-class-method) method, which returns an array of [`Marten::Email`](pathname:///api/0.3/Marten/Emailing/Email.html) instances. For example: + +```crystal +describe MyObject do + describe "#do_something" do + it "sends an email as expected" do + obj = MyObject.new + obj.do_something + + Marten::Spec.delivered_emails.size.should eq 1 + Marten::Spec.delivered_emails[0].subject.should eq "Test subject" + end + end +end +``` + +:::info +Note that Marten also automatically ensures that the collected emails are automatically reset after each spec execution so that you don't have to take care of that directly. +::: diff --git a/docs/versioned_docs/version-0.3/emailing.mdx b/docs/versioned_docs/version-0.3/emailing.mdx new file mode 100644 index 000000000..b69abcd71 --- /dev/null +++ b/docs/versioned_docs/version-0.3/emailing.mdx @@ -0,0 +1,31 @@ +--- +title: Emailing +--- + +import DocCard from '@theme/DocCard'; + +Marten provides a convenient email definition mechanism that leverages templates. These emails are sent through the use of a generic emailing backend system that can be used to fully customize how emails are delivered. + +## Guides + +
+
+ +
+
+ +## How-to's + +
+
+ +
+
+ +## Reference + +
+
+ +
+
diff --git a/docs/versioned_docs/version-0.3/emailing/how-to/create-custom-emailing-backends.md b/docs/versioned_docs/version-0.3/emailing/how-to/create-custom-emailing-backends.md new file mode 100644 index 000000000..43f490b35 --- /dev/null +++ b/docs/versioned_docs/version-0.3/emailing/how-to/create-custom-emailing-backends.md @@ -0,0 +1,30 @@ +--- +title: Create emailing backends +description: How to create custom emailing backends. +--- + +Marten lets you easily create custom [emailing backends](../introduction#emailing-backends) that you can then use as part of your application when it comes to sending emails. + +## Basic backend definition + +Defining an emailing backend is as simple as creating a class that inherits from the [`Marten::Emailing::Backend::Base`](pathname:///api/0.3/Marten/Emailing/Backend/Base.html) abstract class and that implements a unique `#deliver` method. This method takes a single `email` argument (instance of [`Marten::Emailing::Email`](pathname:///api/0.3/Marten/Emailing/Email.html)), corresponding to the email to deliver. + +For example: + +```crystal +class CustomEmailingBackend < Marten::Emailing::Backend::Base + def deliver(email : Email) + # Deliver the email! + end +end +``` + +## Enabling the use of custom emailing backends + +Custom emailing backends can be used by assigning an instance of the corresponding class to the [`emailing.backend`](../../development/reference/settings#backend-1) setting. + +For example: + +```crystal +config.emailing.backend = CustomEmailingBackend.new +``` diff --git a/docs/versioned_docs/version-0.3/emailing/introduction.md b/docs/versioned_docs/version-0.3/emailing/introduction.md new file mode 100644 index 000000000..8f9c4810c --- /dev/null +++ b/docs/versioned_docs/version-0.3/emailing/introduction.md @@ -0,0 +1,138 @@ +--- +title: Introduction to emails +description: Learn how to define emails in a Marten project and how to deliver them. +sidebar_label: Introduction +--- + +Marten lets you define emails in a very declarative way and gives you the ability to fully customize the content of these emails, their properties and associated header values, and obviously how they should be delivered. + +## Email definition + +Emails must be defined as subclasses of the [`Emailing::Email`](pathname:///api/0.3/Marten/Emailing/Email.html) abstract class and they usually live in an `emails` folder at the root of an application. These classes can define to which email addresses the email is sent (including CC or BCC addresses) and with which [templates](../templates) the body of the email (HTML or plain text) is rendered. + +For example, the following snippet defines a simple email that is sent to a specific user's email address: + +```crystal +class WelcomeEmail < Marten::Email + from "no-reply@martenframework.com" + to @user.email + subject "Hello!" + template_name "emails/welcome_email.html" + + def initialize(@user : User) + end +end +``` + +:::info +It is not necessary to systematically specify the `from` email address with the [`#from`](pathname:///api/0.3/Marten/Emailing/Email.html#from(value)-macro) macro. Indeed, unless specified, the default "from" email address that is defined in the [`emailing.from_address`](../development/reference/settings#from_address) setting is automatically used. +::: + +In the above snippet, a `WelcomeEmail` email class is defined by inheriting from the [`Emailing::Email`](pathname:///api/0.3/Marten/Emailing/Email.html) abstract class. This email is initialized with a hypothetical `User` record, and this user's email address is used as the recipient email (through the use of the [`#to`](pathname:///api/0.3/Marten/Emailing/Email.html#to(value)-macro) macro). Other email properties are also defined in the above snippet, such as the "from" email address ([`#from`](pathname:///api/0.3/Marten/Emailing/Email.html#from(value)-macro) macro) and the subject of the email ([`#subject`](pathname:///api/0.3/Marten/Emailing/Email.html#subject(value)-macro) macro). + +### Specifying email properties + +Most email properties (eg. from address, recipient addresses, etc) can be specified in two ways: + +* through the use of a dedicated macro +* by overriding a corresponding method in the email class + +Indeed, it is convenient to define email properties through the use of the dedicated macros: [`#from`](pathname:///api/0.3/Marten/Emailing/Email.html#from(value)-macro) for the sender email, [`#to`](pathname:///api/0.3/Marten/Emailing/Email.html#to(value)-macro) for the recipient addresses, [`#cc`](pathname:///api/0.3/Marten/Emailing/Email.html#cc(value)-macro) for the CC addresses, [`#bcc`](pathname:///api/0.3/Marten/Emailing/Email.html#bcc(value)-macro) for the BCC addresses, [`#reply_to`](pathname:///api/0.3/Marten/Emailing/Email.html#reply_to(value)-macro) for the Reply-To address, and [`#subject`](pathname:///api/0.3/Marten/Emailing/Email.html#subject(value)-macro) for the email subject. + +That being said, if more complicated logics need to be implemented to generate these email properties, it is perfectly possible to simply override the corresponding method in the considered email class. For example: + +```crystal +class WelcomeEmail < Marten::Email + from "no-reply@martenframework.com" + to @user.email + template_name "emails/welcome_email.html" + + def initialize(@user : User) + end + + def subject + if @user.referred? + "Glad to see you here!" + else + "Welcome to the app!" + end + end +end +``` + +### Defining HTML and text bodies + +The HTML body (and optionally text body) of the email is rendered using a [template](../templates) whose name can be specified by using the [`#template_name`](pathname:///api/0.3/Marten/Emailing/Email.html#template_name(template_name%3AString%3F%2Ccontent_type%3AContentType|String|Symbol%3DContentType%3A%3AHTML)%3ANil-class-method) class method. By default, unless explicitly specified, it is assumed that the template specified to this method is used for rendering the HTML body of the email. That being said, it is possible to explicitly specify for which content type the template should be used by specifying an optional `content_type` argument as follows: + +```crystal +class WelcomeEmail < Marten::Email + to @user.email + subject "Hello!" + template_name "emails/welcome_email.html", content_type: :html + template_name "emails/welcome_email.txt", content_type: :text + + def initialize(@user : User) + end +end +``` + +Note that it is perfectly valid to specify one template for rendering the HTML body AND another one for rendering the text body (like in the above example). + +:::info +Note that you can define [`#html_body`](pathname:///api/0.3/Marten/Emailing/Email.html#html_body%3AString%3F-instance-method) and [`#text_body`](pathname:///api/0.3/Marten/Emailing/Email.html#html_body%3AString%3F-instance-method) methods if you need to override the logic that allows generating the HTML or text body of your email. +::: + +### Defining custom headers + +If you need to insert custom headers into your emails, then you can easily do so by defining a `#headers` method in your email class. This method must return a hash of string keys and values. + +For example: + +```crystal +class WelcomeEmail < Marten::Email + to @user.email + template_name "emails/welcome_email.html" + + def initialize(@user : User) + end + + def headers + {"X-Foo" => "bar"} + end +end +``` + +## Sending emails + +Emails are sent _synchronously_ through the use of the [`#deliver`](pathname:///api/0.3/Marten/Emailing/Email.html#deliver-instance-method). For example, the `WelcomeEmail` email defined in the previous sections could be initialized and delivered by doing: + +```crystal +email = WelcomeEmail.new(user) +email.deliver +``` + +When calling [`#deliver`](pathname:///api/0.3/Marten/Emailing/Email.html#deliver-instance-method), the considered email will be delivered by using the currently configured [emailing backend](#emailing-backends). + +## Emailing backends + +Emailing backends define _how_ emails are actually sent when [`#deliver`](pathname:///api/0.3/Marten/Emailing/Email.html#deliver-instance-method) gets called. For example, a [development backend](./reference/backends#development-backend) might simply "collect" the sent emails and print their information to the standard output. Other backends might also integrate with existing email services or interact with an SMTP server to ensure email delivery. + +Which backend is used when sending emails is something that is controlled by the [`emailing.backend`](../development/reference/settings#backend-1) setting. All the available emailing backends are listed in the [emailing backend reference](./reference/backends). + +:::tip +If necessary, it is also possible to override which emailing backend is used on a per-email basis by leveraging the [`#backend`](pathname:///api/0.3/Marten/Emailing/Email.html#backend(backend%3ABackend%3A%3ABase)%3ANil-class-method) class method. For example: + +```crystal +class WelcomeEmail < Marten::Email + from "no-reply@martenframework.com" + to @user.email + subject "Hello!" + template_name "emails/welcome_email.html" + + backend Marten::Emailing::Backend::Development.new(print_emails: true) + + def initialize(@user : User) + end +end +``` +::: diff --git a/docs/versioned_docs/version-0.3/emailing/reference/backends.md b/docs/versioned_docs/version-0.3/emailing/reference/backends.md new file mode 100644 index 000000000..742e066e5 --- /dev/null +++ b/docs/versioned_docs/version-0.3/emailing/reference/backends.md @@ -0,0 +1,31 @@ +--- +title: Emailing backends +description: Emailing backends reference. +sidebar_label: Backends +--- + +## Built-in backends + +### Development backend + +This is the default backend used as part of the [`emailing.backend`](../../development/reference/settings#backend-1) setting. + +This backend "collects" all the emails that are "delivered", which can be used in specs in order to test sent emails. This "collect" behavior can be disabled if necessary, and the backend can also be configured to print email details to the standard output. + +For example: + +```crystal +config.emailing.backend = Marten::Emailing::Backend::Development.new(print_emails: true, collect_emails: false) +``` + +## Other backends + +Additional emailing backend shards are also maintained under the umbrella of the Marten project or by the community itself and can be used as part of your application depending on your specific email sending requirements: + +* [`marten-smtp-emailing`](https://github.com/martenframework/marten-smtp-emailing) provides an SMTP emailing backend +* [`marten-sendgrid-emailing`](https://github.com/martenframework/marten-sendgrid-emailing) provides a [Sendgrid](https://sendgrid.com/) emailing backend +* [`marten-mailgun-emailing`](https://github.com/martenframework/marten-mailgun-emailing) provides a [Mailgun](https://www.mailgun.com/) emailing backend + +:::info +Feel free to contribute to this page and add links to your shards if you've created emailing backends that are not listed here! +::: diff --git a/docs/versioned_docs/version-0.3/files.mdx b/docs/versioned_docs/version-0.3/files.mdx new file mode 100644 index 000000000..1ec2b505c --- /dev/null +++ b/docs/versioned_docs/version-0.3/files.mdx @@ -0,0 +1,26 @@ +--- +title: Files +--- + +import DocCard from '@theme/DocCard'; + +Dealing with files is a common requirement for web applications. This section covers how to upload and manage files. + +## Guides + +
+
+ +
+
+ +
+
+ +## How-to's + +
+
+ +
+
diff --git a/docs/versioned_docs/version-0.3/files/how-to/create-custom-file-storages.md b/docs/versioned_docs/version-0.3/files/how-to/create-custom-file-storages.md new file mode 100644 index 000000000..c7e189e6c --- /dev/null +++ b/docs/versioned_docs/version-0.3/files/how-to/create-custom-file-storages.md @@ -0,0 +1,78 @@ +--- +title: Create custom file storages +description: Learn how to create custom file storages. +--- + +Marten uses a file storage mechanism to perform file operations like saving files, deleting files, generating URLs, ... This file storages mechanism allows to save files in different backends by leveraging a standardized API. You can leverage this capability to implement custom file storages (which you can then use for [assets](../../assets/introduction) or as part of [file model fields](../uploading-files#persisting-uploaded-files-in-model-records)). + +## Basic file storage implementation + +File storages are implemented as subclasses of the [`Marten::Core::Storage::Base`](pathname:///api/0.3/Marten/Core/Storage/Base.html) abstract class. As such, they must implement a set of mandatory methods which provide the following functionalities: + +* saving files ([`#save`](pathname:///api/0.3/Marten/Core/Storage/Base.html#save(filepath%3AString%2Ccontent%3AIO)%3AString-instance-method)) +* deleting files ([`#delete`](pathname:///api/0.3/Marten/Core/Storage/Base.html#delete(filepath%3AString)%3ANil-instance-method)) +* opening files ([`#open`](pathname:///api/0.3/Marten/Core/Storage/Base.html#open(filepath%3AString)%3AIO-instance-method)) +* verifying that files exist ([`#exist?`](pathname:///api/0.3/Marten/Core/Storage/Base.html#exists%3F(filepath%3AString)%3ABool-instance-method)) +* retrieving file sizes ([`#size`](pathname:///api/0.3/Marten/Core/Storage/Base.html#size(filepath%3AString)%3AInt64-instance-method)) +* retrieving file URLs ([`#url`](pathname:///api/0.3/Marten/Core/Storage/Base.html#url(filepath%3AString)%3AString-instance-method)) + +Note that you can fully customize how file storage objects are initialized. + +For example, a custom-made "file system" storage (that reads and writes files in a specific folder of the local file system) could be implemented as follows: + +```crystal +require "file_utils" + +class FileSystem < Marten::Core::Storage::Base + def initialize(@root : String, @base_url : String) + end + + def delete(filepath : String) : Nil + File.delete(path(filepath)) + rescue File::NotFoundError + raise Marten::Core::Storage::Errors::FileNotFound.new("File '#{filepath}' cannot be found") + end + + def exists?(filepath : String) : Bool + File.exists?(path(filepath)) + end + + def open(filepath : String) : IO + File.open(path(filepath), mode: "rb") + rescue File::NotFoundError + raise Marten::Core::Storage::Errors::FileNotFound.new("File '#{filepath}' cannot be found") + end + + def size(filepath : String) : Int64 + File.size(path(filepath)) + end + + def url(filepath : String) : String + File.join(base_url, URI.encode_path(filepath)) + end + + def write(filepath : String, content : IO) : Nil + new_path = path(filepath) + + FileUtils.mkdir_p(Path[new_path].dirname) + + File.open(new_path, "wb") do |new_file| + IO.copy(content, new_file) + end + end + + private getter root + private getter base_url + + private def path(filepath) + File.join(root, filepath) + end +end +``` + +## Using custom file storages + +You have many options when it comes to using your custom file storage classes, and those depend on what you are trying to do: + +* if you want to use a custom storage for [assets](../../assets/introduction), then you will likely want to assign an instance of your custom storage class to the [`assets.storage`](../../development/reference/settings#storage) setting (see [Assets storage](../../assets/introduction#assets-storage) to learn more about assets storages specifically) +* if you want to use a custom storage for all your [file model fields](../../models-and-databases/reference/fields#file), then you will likely want to assign an instance of your custom storage class to the [`media_files.storage`](../../development/reference/settings#storage-1) setting (see [File storages](../managing-files#file-storages) to learn more about file storages specifically) diff --git a/docs/versioned_docs/version-0.3/files/managing-files.md b/docs/versioned_docs/version-0.3/files/managing-files.md new file mode 100644 index 000000000..749eaa380 --- /dev/null +++ b/docs/versioned_docs/version-0.3/files/managing-files.md @@ -0,0 +1,226 @@ +--- +title: Managing files +description: Learn how to manage uploaded files. +sidebar_label: Managing files +--- + +Marten gives you the ability to associate uploaded files with model records and to fully customize how and where those files are persisted. This section covers the basics of using files with models, how to interact with file objects, and introduces the concept of file storage. + +## Using files with models + +You can make use of the [`file`](../models-and-databases/reference/fields#file) field when defining models: this allows to associate an uploaded file with specific model records. + +For example, let's consider the following model: + +```crystal +class Attachment < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :file, :file, blank: false, null: false +end +``` + +Any `Attachment` model record will have a `file` attribute allowing interacting with the attached file: + +```crystal +attachment = Attachment.first! +attachment.file # => # +attachment.file.attached? # => true +attachment.file.name # => "test.txt" +attachment.file.size # => 5796929 +attachment.file.url # => "/media/test.txt" +``` + +The object returned by the `Attachment#file` method is a "file object": an instance of [`Marten::DB::Field::File::File`](pathname:///api/0.3/Marten/DB/Field/File/File.html). These objects and their associated capabilities are described below in [File objects](#file-objects). + +:::tip Under which path are files persisted? +Files are stored at the root of the media [storage](#file-storages) by default. It should be noted that the path used to persist files in storages can be configured by setting the `upload_to` [`file`](../models-and-databases/reference/fields#file) field option. + +For example, the previous `Attachment` model could be rewritten as follows to ensure that files are persisted in a `foo/bar` folder: + +```crystal +class Attachment < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :file, :file, blank: false, null: false, upload_to: "foo/bar" +end +``` + +It should also be noted that `upload_to` can correspond to a proc that takes the name of the file to save, which can be used to implement more complex file path generation logic if necessary: + +```crystal +class Attachment < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :file, :file, blank: false, null: false, upload_to: ->(name : String) { File.join("files/uploads", name) } +end +``` +::: + +It should be noted that saving a model record will automatically result in any associated files being saved and persisted in the right [storage](#file-storages) automatically. For example, the following snippet reads a locally available file and attaches it to a new model record: + +```crystal +attachment = Attachment.new + +File.open("test.txt") do |file| + attachment.file = file + attachment.save! +end +``` + +:::info +You don't need to take care of possible collisions between attached file names: Marten automatically ensures that uploaded files have a unique file name in the destination storage in order to avoid possible conflicts. +::: + +## File objects + +As mentioned previously, file objects are used internally by Marten to allow interacting with files that are associated with model records. These objects are instances of the [`Marten::DB::Field::File::File`](pathname:///api/0.3/Marten/DB/Field/File/File.html) class. They give access to basic file properties and they allow to interact with the associated IO. + +It should be noted that these "file objects" are **always** associated with a model record (persisted or not), and as such, they are only used in the context of the [`file`](../models-and-databases/reference/fields#file) model field. + +Finally, it's worth mentioning that file objects can be **attached** and/or **committed**: + +* an **attached** file object has an associated file set: in that case, its [`#attached?`](pathname:///api/0.3/Marten/DB/Field/File/File.html#attached%3F-instance-method) method returns `true` +* a **committed** file object has an associated file that is _persisted_ to the underlying [storage](#file-storages): in that case, its [`#committed?`](pathname:///api/0.3/Marten/DB/Field/File/File.html#committed%3F%3ABool-instance-method) method returns `true` + +For example: + +```crystal +attachment = Attachment.last! +attachment.file.attached? # => true +attachment.file.committed? # => true +``` + +### Accessing file properties + +File objects give access to basic file properties through the use of the following methods: + +| Method | Description | +| ----------- | ----------- | +| `#file` | Returns the associated / "wrapped" file object. This can be a real [`File`](https://crystal-lang.org/api/File.html) object, an uploaded file (instance of [`Marten::HTTP::UploadedFile`](pathname:///api/0.3/Marten/HTTP/UploadedFile.html)), or `nil` if no file is associated yet. | +| `#name` | Returns the name of the file. | +| `#size` | Returns the size of the file, using the associated [storage](#file-storages). | +| `#url` | Returns the URL of the file, using the associated [storage](#file-storages). | + +### Accessing the underlying file content + +File objects allow you to access the underlying file content through the use of the [`#open`](pathname:///api/0.3/Marten/DB/Field/File/File.html#open%3AIO-instance-method) method. This method returns an [`IO`](https://crystal-lang.org/api/IO.html) object. + +For example: + +```crystal +attachment = Attachment.last! +file_io = attachment.file.open +puts file_io.gets_to_end +``` + +### Updating the attached file + +It is possible to update the actual file of a "file object" by using the [`#save`](pathname:///api/0.3/Marten/DB/Field/File/File.html#save(filepath%3A%3A%3AString%2Ccontent%3AIO%2Csave%3Dfalse)%3ANil-instance-method) method. This method allows saving the content of a specified [`IO`](https://crystal-lang.org/api/IO.html) object and associating it with a specific file path in the underlying [storage](#file-storages). + +For example: + +```crystal +attachment = Attachment.new + +File.open("test.txt") do |file| + attachment.file.save("path/to/test.txt", file) + attachment.save! +end + +attachment.file.url # => "/media/path/to/test.txt" +``` + +### Deleting the attached file + +It is also possible to manually "delete" the file associated with the "file object". To do so, the [`#delete`](pathname:///api/0.3/Marten/DB/Field/File/File.html#delete(save%3Dfalse)%3ANil-instance-method) method can be used. It should be noted that calling this method will remove the association between the model record and the file AND will also delete the file in the considered [storage](#file-storages). + +For example: + +```crystal +attachment = Attachment.last! +attachment.file.delete +attachment.file.attached? # => false +attachment.file.committed? # => false +``` + +## File storages + +Marten uses a file storage mechanism to perform file operations like saving files, deleting files, generating URLs, ... This file storages mechanism allows to save files in different backends by leveraging a standardized API (eg. in the local file system, in a cloud bucket, etc). + +By default, [`file`](../models-and-databases/reference/fields#file) model fields make use of the configured "media" storage. This storage uses the [`settings.media_files`](../development/reference/settings#media-files-settings) settings to determine what storage backend to use, and where to persist files. By default, the media storage uses the [`Marten::Core::Store::FileSystem`](pathname:///api/0.3/Marten/Core/Storage/FileSystem.html) storage backend, which ensures that files are persisted in the local file system, where the Marten application is running. + +### Interacting with the media file storage + +You won't usually need to interact directly with the file storage, but it's worth mentioning that storage objects share the same API. Indeed, the class of these storage objects must inherit from the [`Marten::Core::Storage::Base`](pathname:///api/0.3/Marten/Core/Storage/Base.html) abstract class and implement a set of mandatory methods which provide the following functionalities: + +* saving files ([`#save`](pathname:///api/0.3/Marten/Core/Storage/Base.html#save(filepath%3AString%2Ccontent%3AIO)%3AString-instance-method)) +* deleting files ([`#delete`](pathname:///api/0.3/Marten/Core/Storage/Base.html#delete(filepath%3AString)%3ANil-instance-method)) +* opening files ([`#open`](pathname:///api/0.3/Marten/Core/Storage/Base.html#open(filepath%3AString)%3AIO-instance-method)) +* verifying that files exist ([`#exist?`](pathname:///api/0.3/Marten/Core/Storage/Base.html#exists%3F(filepath%3AString)%3ABool-instance-method)) +* retrieving file sizes ([`#size`](pathname:///api/0.3/Marten/Core/Storage/Base.html#size(filepath%3AString)%3AInt64-instance-method)) +* retrieving file URLs ([`#url`](pathname:///api/0.3/Marten/Core/Storage/Base.html#url(filepath%3AString)%3AString-instance-method)) + +These capabilities are highlighted with the following example, where the media storage is used to interact with files: + +```crystal +file = File.open("test.txt") +storage = Marten.media_files_storage + +filepath = storage.save("test.txt", file) +storage.exists?(filepath) # => true +storage.exists?("unknown") # => false +storage.size(filepath) # => 13 +storage.url(filepath) # => "/media/test_c43ba020.txt" +storage.delete(filepath) # => nil +storage.exists?(filepath) # => false +``` + +It should be noted that everything in the previous example could be done with a custom storage initialized manually as well: + +```crystal +file = File.open("test.txt") +storage = Marten::Core::Storage::FileSystem.new(root: "/tmp", base_url: "/tmp") + +filepath = storage.save("test.txt", file) +storage.exists?(filepath) # => true +storage.exists?("unknown") # => false +storage.size(filepath) # => 13 +storage.url(filepath) # => "/tmp/test.txt" +storage.delete(filepath) # => nil +storage.exists?(filepath) # => false +``` + +### Using a different storage with models + +As mentioned previously, [`file`](../models-and-databases/reference/fields#file) model fields make use of the configured "media" storage by default. That being said, it is possible to leverage the `storage` option in order to make use of another storage if necessary. + +For example: + +```crystal +custom_storage = Marten::Core::Storage::FileSystem.new(root: "/tmp", base_url: "/tmp") + +class Attachment < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :file, :file, blank: false, null: false, storage: custom_storage +end +``` + +When doing this, all the file operations will be done using the configured storage instead of the default media storage. + +## Serving uploaded files during development + +Marten provides a handler that you can use to serve media files in development environments only. This handler ([`Marten::Handlers::Defaults::Development::ServeMediaFile`](pathname:///api/0.3/Marten/Handlers/Defaults/Development/ServeMediaFile.html)) is automatically mapped to a route when creating new projects through the use of the [`new`](../development/reference/management-commands#new) management command: + +```crystal +Marten.routes.draw do + # Other routes... + + if Marten.env.development? + path "#{Marten.settings.media_files.url}", Marten::Handlers::Defaults::Development::ServeMediaFile, name: "media_file" + end +end +``` + +As you can see, this route will automatically use the URL that is configured as part of the [`url`](../development/reference/settings#url-1) media files setting. For example, this means that a `foo/bar.txt` media file would be served by the `/media/foo/bar.txt` route in development if the [`url`](../development/reference/settings#url-1) setting is set to `/media/`. + +:::warning +It is very important to understand that this handler should **only** be used in development environments. Indeed, the [`Marten::Handlers::Defaults::Development::ServeMediaFile`](pathname:///api/0.3/Marten/Handlers/Defaults/Development/ServeMediaFile.html) handler is not suited for production environments as it is not really efficient or secure. A better way to serve uploaded files is to leverage a web server or a cloud bucket for example (depending on the configured media files storage). +::: diff --git a/docs/versioned_docs/version-0.3/files/uploading-files.md b/docs/versioned_docs/version-0.3/files/uploading-files.md new file mode 100644 index 000000000..594f435f4 --- /dev/null +++ b/docs/versioned_docs/version-0.3/files/uploading-files.md @@ -0,0 +1,87 @@ +--- +title: Uploading files +description: Learn how to upload files. +sidebar_label: Uploading files +--- + +Marten gives you the ability to interact with uploaded files. These files are made available with each HTTP request object, and it is also possible to validate them using schemas. The following document explains how to expect and manipulate uploaded files, and what are their associated characteristics. + +## Accessing uploaded files + +Uploaded files are made available in the [`#data`](pathname:///api/0.3/Marten/HTTP/Request.html#data%3AParams%3A%3AData-instance-method) hash-like object of any HTTP request object (instance of [`Marten::HTTP::Request`](pathname:///api/0.3/Marten/HTTP/Request.html)). These file objects are instances of the [`Marten::HTTP::UploadedFile`](pathname:///api/0.3/Marten/HTTP/UploadedFile.html) class. + +For example, you could access and process a `file` file originating from an HTML form using a handler like this: + +```crystal +class ProcessUploadedFileHandler < Marten::Handler + def post + file = request.data["file"].as(Marten::HTTP::UploadedFile) + respond "Processed file: #{file.filename}" + end +end +``` + +[`Marten::HTTP::UploadedFile`](pathname:///api/0.3/Marten/HTTP/UploadedFile.html) objects give you access to the following key methods, which allow you to interact with the uploaded file and its content: + +* `#filename` returns the name of the uploaded file +* `#size` returns the size of the uploaded file +* `#io` returns a regular [`IO`](https://crystal-lang.org/api/IO.html) object allowing to read the content of the file and interact with it + +:::info Where are uploaded files stored? +All uploaded files are automatically persisted to a temporary file in the system's temporary directory (usually this corresponds to the `/tmp` folder). +::: + +## Expecting uploaded files with schemas + +If you use [schemas](../schemas/introduction) to validate input data (such as form data), then it's worth noting that you can explicitly define that you expect files in the validated data. The simplest way to do that is to make use of the [`file`](../schemas/reference/fields#file) schema field. + +For example, you could define the following schema: + +```crystal +class UploadFileSchema < Marten::Schema + field :file, :file +end +``` + +And use it in a regular [schema generic handler](../handlers-and-http/reference/generic-handlers#processing-a-schema) like this: + +```crystal +class UploadFileHandler < Marten::Handlers::Schema + schema UploadFileSchema + template_name "upload_file.html" + success_url "/" + + def process_valid_schema + file = schema.validated_data["file"] + # Do something with the uploaded file... + + super + end +end +``` + +The presence/absence of the file (and - optionally - some of its attributes) will be validated according to the schema definition when `POST` requests are processed by the handler. + +## Persisting uploaded files in model records + +Models can define [`file`](../models-and-databases/reference/fields#file) fields and persist "references" of uploaded files in their rows. This allows "retaining" specific uploaded files and associating their references with specific model records. + +For example, we could modify the handler in the previous section so that it persists and associate the uploaded file to a new `Attachment` record as follows: + +```crystal +class UploadFileHandler < Marten::Handlers::Schema + schema UploadFileSchema + template_name "upload_file.html" + success_url "/" + + def process_valid_schema + file = schema.validated_data["file"] + // highlight-next-line + Attachment.create!(file: file) + + super + end +end +``` + +Here, the `UploadFileHandler` inherits from the [`Marten::Handlers::Schema`](pathname:///api/0.3/Marten/Handlers/Schema.html) generic handler. It would also make sense to leverage the [`Marten::Handlers::RecordCreate`](pathname:///api/0.3/Marten/Handlers/RecordCreate.html) generic handler to process the schema and create the `Attachment` record at the same time. diff --git a/docs/versioned_docs/version-0.3/getting-started.mdx b/docs/versioned_docs/version-0.3/getting-started.mdx new file mode 100644 index 000000000..a9ba7ca68 --- /dev/null +++ b/docs/versioned_docs/version-0.3/getting-started.mdx @@ -0,0 +1,18 @@ +--- +title: Getting Started +--- + +import DocCard from '@theme/DocCard'; + +If you are new to Marten or to web development, the following resources will help you get started. + +## Guides + +
+
+ +
+
+ +
+
diff --git a/docs/versioned_docs/version-0.3/getting-started/installation.md b/docs/versioned_docs/version-0.3/getting-started/installation.md new file mode 100644 index 000000000..4b5ad9dc0 --- /dev/null +++ b/docs/versioned_docs/version-0.3/getting-started/installation.md @@ -0,0 +1,86 @@ +--- +title: Installation +description: Get started by installing Marten and its dependencies. +--- + + +This guide will help you get started in order to install Marten and its dependencies. Let's get started! + +## Install Crystal + +Marten is a Crystal web framework; as such Crystal must be installed on your system. There are many ways to install Crystal, but we'll only highlight what we think are the most common ones here for the sake of simplicity: using Homebrew (macOS or Linux) or the APT package manager (Ubuntu, Debian). Please refer to the official [Crystal installation guide](https://crystal-lang.org/install/) if these methods don't work for you. + +### Using Homebrew + +On macOS or Linux, Crystal can be installed using [Homebrew](https://brew.sh/) (also known as Linuxbrew) by running the following command: + +```bash +brew install crystal +``` + +### Using APT + +On Ubuntu, Debian or any other Linux distribution using the APT package manager, Crystal can be installed by running the following command: + +```bash +curl -fsSL https://crystal-lang.org/install.sh | sudo bash +``` + +## Install a database + +New Marten projects will use a SQLite database by default: this lightweight serverless database application is usually already pre-installed on most of the existing operating systems, which makes it an ideal candidate for a development or a testing database. As such, if you choose to use SQLite for your new Marten project, you can very probably skip this section. + +Marten also has built-in support for PostgreSQL and MySQL. Please refer to the applicable official documentation to install your database of choice: + +* [PostgreSQL Installation Guide](https://wiki.postgresql.org/wiki/Detailed_installation_guides) +* [MySQL Installation Guide](https://dev.mysql.com/doc/refman/8.0/en/installing.html) +* [SQLite Installation Guide](https://www.tutorialspoint.com/sqlite/sqlite_installation.htm) + +Each database requires the use of a dedicated shard (package of Crystal code). You don't have to install any of these right now if you are just starting with the framework or if you are planning to follow the [tutorial](./tutorial), but you may have to add one of the following entries to your project's `shard.yml` file later: + +* [crystal-pg](https://github.com/will/crystal-pg) (needed when using PostgreSQL databases) +* [crystal-mysql](https://github.com/crystal-lang/crystal-mysql) (needed when using MySQL databases) +* [crystal-sqlite3](https://github.com/crystal-lang/crystal-sqlite3) (needed when using SQLite3 databases) + +## Install Marten + +The next step is to install the Marten CLI. This tool will let you easily generate new Marten projects or applications. + +### Using Homebrew + +On macOS or Linux, Marten can be installed using [Homebrew](https://brew.sh/) (also known as Linuxbrew) by running the following commands: + +```bash +brew tap martenframework/marten +brew install marten +``` + +Once the installation is complete, you should be able to use the `marten` command: + +```bash +marten -v +``` + +### From the sources + +Marten can be installed from the sources by running the following commands: + +```bash +git clone https://github.com/martenframework/marten +cd marten +shards install +crystal build src/marten_cli.cr -o bin/marten +mv bin/marten /usr/local/bin +``` + +Once the above steps are done, you should be able to verify that the `marten` command works as expected by running: + +```bash +marten -v +``` + +## Next steps + +_Congrats! You’re in._ + +You can now move on to the [introduction tutorial](./tutorial). diff --git a/docs/versioned_docs/version-0.3/getting-started/tutorial.md b/docs/versioned_docs/version-0.3/getting-started/tutorial.md new file mode 100644 index 000000000..67102308d --- /dev/null +++ b/docs/versioned_docs/version-0.3/getting-started/tutorial.md @@ -0,0 +1,888 @@ +--- +title: Tutorial +description: Learn how to use Marten by creating a simple web application. +--- + +This guide will walk you through the creation of a simple weblog application, which will help you learn the basics of the Marten web framework. It is designed for beginners who want to get started by creating a Marten web project, so no prior experience with the framework is required. + +## Requirements + +This guide assumes that [Crystal and the Marten CLI are properly installed already](./installation.md). You can verify that the Marten CLI is properly installed by running the following command: + +```bash +marten -v +``` + +This should output the version of your Marten installation. + +## What is Marten? + +Marten is a web application framework written in the Crystal programming language. It is designed to make developing web applications easy and fun; and it does so by making some assumptions regarding the common needs that developers may encounter when building web applications. + +## Creating a project + +Creating a project is the first thing to do in order to start working on a Marten web application. This creation process can be done through the use of the `marten` command, and it will ensure that the basic structure of a Marten project is properly generated. + +This can be achieved from the command line, where you can run the following command to create your first Marten project: + +```bash +marten new project myblog +``` + +The above command will create a `myblog` directory inside your current directory. This new folder should have the following content: + +``` +myblog/ +├── config +│   ├── initializers +│   ├── settings +│   │   ├── base.cr +│   │   ├── development.cr +│   │   ├── production.cr +│   │   └── test.cr +│   └── routes.cr +├── spec +│   └── spec_helper.cr +├── src +│   ├── assets +│   ├── handlers +│   ├── migrations +│   ├── models +│   ├── schemas +│   ├── templates +│   ├── cli.cr +│   ├── project.cr +│   └── server.cr +├── manage.cr +└── shard.yml +``` + +These files and folders are described below: + +| Path | Description | +| ----------- | ----------- | +| config/ | Contains the configuration of the project. This includes environment-specific Marten configuration settings, initializers, and web application routes. | +| spec/ | Contains the project specs, allowing you to test your application. | +| src/ | Contains the source code of the application. By default this folder will include a `project.cr` file (where all dependencies - including Marten itself - are required), a `server.cr` file (which starts the Marten web server), a `cli.cr` file (where migrations and CLI-related abstractions are required), and `assets`, `handlers`, `migrations`, `models`, `schemas`, and `templates` folders. | +| .gitignore | Regular `.gitignore` file which tells git the files and directories that should be ignored. | +| manage.cr | This file defines a CLI that lets you interact with your Marten project in order to perform various actions (e.g. running database migrations, collecting assets, etc). | +| shard.yml | The standard [shard.yml](https://crystal-lang.org/reference/the_shards_command/index.html) file, that lists the dependencies that are required to build your application. | + +Now that the project structure is created, you can change into the `myblog` directory (if you haven't already) in order to install the project dependencies by running the following command: + +```bash +shards install +``` + +:::info +Marten projects are organized around the concept of "apps". A Marten app is a set of abstractions (usually defined under a unique folder) that contributes specific behaviours to a project. For example, apps can provide [models](../models-and-databases) or [handlers](../handlers-and-http). They allow to separate a project into a set of logical and reusable components. Another interesting benefit of apps is that they can be extracted and distributed as external shards. This pattern allows third-party libraries to easily contribute models, migrations, handlers, or templates to other projects. The use of apps is activated by simply adding app classes to the [`installed_apps`](../development/reference/settings#installed_apps) setting. + +By default, when creating a new project through the use of the [`new`](../development/reference/management-commands#new) command, no explicit app will be created nor installed. This is because each Marten project comes with a default "main" app that corresponds to your standard `src` folder. Models, migrations, or other classes defined in this folder are associated with the main app by default, unless they are part of another explicitly defined application. + +As projects grow in size and scope, it is generally encouraged to start thinking in terms of apps and how to split models, handlers, or features across multiple apps depending on their intended responsibilities. Please refer to [Applications](../development/applications) to learn more about applications and how to structure your projects using them. +::: + +## Running the development server + +Now that you have a fully functional web project, you can start a development server by using the following command: + +```bash +marten serve +``` + +This will start a Marten development server. To verify that it's working as expected, you can open a browser and navigate to [http://localhost:8000](http://localhost:8000). When doing so, you should be greeted by the Marten "welcome" page: + +![Marten welcome page](../static/img/getting-started/tutorial/marten_welcome_page.png) + +:::info +Your project development server will automatically be available on the internal IP at port 8000. The server port and IP can be changed easily by modifying the `config/settings/development.cr` file: + +```crystal +Marten.configure :development do |config| + config.debug = true + config.host = "localhost" + config.port = 3000 +end +``` +::: + +Once started, the development server will watch your project source files and will automatically recompile them when they are updated; it will also take care of restarting your project server. As such, you don't have to manually restart the server when making changes to your application source files. + +## Writing a first handler + +Let's start by creating the first handler for your project. To do so, create a `src/handlers/home_handler.cr` file with the following content: + +```crystal title="src/handlers/home_handler.cr" +class HomeHandler < Marten::Handler + def get + respond("Hello World!") + end +end +``` + +Handlers are classes that process a web request in order to produce a web response. This response can be rendered HTML content or a redirection for example. + +In the above example, the `HomeHandler` handler explicitly processes a `GET` HTTP request and returns a very simple `200 OK` response containing a short text. But in order to access this handler via a browser, it is necessary to map it to a URL route. To do so, you can edit the `config/routes.cr` file as follows: + +```crystal title="config/routes.cr" +Marten.routes.draw do + // highlight-next-line + path "/", HomeHandler, name: "home" + // highlight-next-line + + if Marten.env.development? + path "#{Marten.settings.assets.url}", Marten::Handlers::Defaults::Development::ServeAsset, name: "asset" + path "#{Marten.settings.media_files.url}", Marten::Handlers::Defaults::Development::ServeMediaFile, name: "media_file" + end +end +``` + +The `config/routes.cr` file was automatically created earlier when you initialized the project structure. By using the `#path` method you wired the `HomeHandler` into the routes configuration. + +The `#path` method accepts three arguments: + +* the first argument is the route pattern, which is a string like `/foo/bar`. When Marten needs to resolve a route, it starts at the beginning of the routes array and compares each of the configured routes until it finds a matching one +* the second argument is the handler class associated with the specified route. When a request URL is matched to a specific route, Marten executes the handler that is associated with it +* the last argument is the route name. This is an identifier that can later be used in your codebase to generate the full URL for a specific route, and optionally inject parameters in it + +Now if you go to [http://localhost:8000](http://localhost:8000), you will get the `Hello World!` response that is generated by the handler you just wrote. + +:::tip +Multiple routes can map to the same handler class if necessary. +::: + +:::info +Please refer to [Routing](../handlers-and-http/routing) to learn more about Marten's routing mechanism. +::: + +## Creating the Article model + +[Models](../models-and-databases/introduction) are classes that define what data can be persisted and manipulated by a Marten application. They explicitly specify fields and rules that map to database tables and columns. Model records can be queried and interacted with through a mechanism called [Query sets](../models-and-databases/queries). + +Let's define an `Article` model, which is the linchpin of any weblog application. To do set, let's create a `src/models/article.cr` file with the following content: + +```crystal title="src/models/article.cr" +class Article < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :title, :string, max_size: 255 + field :content, :text +end +``` + +As you can see, Marten models are defined as subclasses of the `Marten::Model` base class, and they explicitly define "fields" through the use of a `field` macro. + +In its current state, our `Article` model contains the following three fields: + +* `id` is a big integer that will hold the unique identifier of an article (primary key of the underlying table record) +* `title` is a string column (`VARCHAR(255)`) that will hold the title of an article +* `content` is a text column (`TEXT`) that will hold the textual content of an article + +## Generating and running migrations + +Our `Article` model above is defined but is not "applied" at the database level yet. In order to create the corresponding table and columns, we will need to generate a [migration](../models-and-databases/migrations) for it. + +Marten provides a migrations mechanism that is designed to be automatic: this means that migrations will be automatically derived from your model definitions. This allows to ensure that the definition of your model and its fields (and underlying columns) is done in one place only, which helps keep your project DRY. + +In order to generate the migration file for the model we created previously, all we need to do is to run the following Marten command: + +```shell +marten genmigrations +``` + +This will output something along those lines: + +```shell +Generating migrations for app 'main': + › Creating [src/migrations/202208072015231_create_main_article_table.cr]... DONE + ○ Create main_article table +``` + +The `genmigrations` command is a way to tell Marten to introspect your project in order to identify whether you added, removed, or modified models. The changes identified by this command are persisted in migration files (living under `migrations/` folders in each Marten application), that you can run later on in order to apply them at the database level. + +Now that we have generated a migration file for our `Article` model, we can apply it at the database level by running the following command: + +```shell +marten migrate +``` + +Which will output the following content: + +```shell +Running migrations: + › Applying main_202208072015231_create_main_article_table... DONE +``` + +The `migrate` command will identify all the migration files that weren't applied to your database yet, and will run them one by one. By doing so, Marten will ensure that the changes you made to your model definitions are applied at the database level, in the corresponding tables. + +:::info +Please refer to [Migrations](../models-and-databases/migrations) to learn more about migrations. +::: + +:::note +For new projects, Marten uses a SQLite database by default. In our case, if we look at the `config/settings/base.cr` file, we can see that the current database configuration looks something like this: + +```crystal +config.database do |db| + db.backend = :sqlite + db.name = Path["myblog.db"].expand +end +``` + +An SQLite database is a good choice in order to try Marten and experiment with it (since SQLite is already pre-installed on most systems). That being said, if you need to use another database backend (for example, PostgreSQL or MySQL), feel free to have a look at the [databases configuration reference](../development/reference/settings#database-settings). +::: + +## Interacting with model records + +Now that our `Article` model table has been created at the database level, let's try to make use of the Marten ORM to create and query article records. + +To do so, we can launch the [Crystal playground](https://crystal-lang.org/reference/master/using_the_compiler/index.html#crystal-play) as follows: + +```shell +crystal play +``` + +You should be able to navigate to [http://localhost:8080](http://localhost:8080) and see a Crystal editor. Once you are there, replace the content of the live editor with the following: + +```crystal +require "./src/project" +Marten.setup +``` + +These lines basically require your project dependencies and ensure that Marten is properly set up. You should keep those in the editor when playing with the following examples. Each of the following snippets is assumed to be copied/pasted below the previous one. The output of these examples is highlighted next to the `# =>` comment line. + +Let's start by initializing a new `Article` object: + +```crystal +article = Article.new(title: "My article", content: "This is my article.") +# => # +``` + +As you can see, by using `#new`, we are initializing a new `Article` object by specifying its field values (`title` and `content`). It should be noted that so far, the object is only _initialized_ and is not saved to the database yet (this is why `id` is set to `nil` in the above snippet). In order to persist the new object at the database level, you can make use of the `#save` method: + +```crystal +article.save +# => true +``` + +The output of the `#save` method is a boolean indicating the result of the object validation: in our case, `true` means that the `Article` object was successfully validated and that the corresponding record was created at the database level. + +Now if we inspect the `article` object again, we should observe that an `id` has been set for the record at hand: + +```crystal +article +# => # +``` + +If we want to fetch this record from the database again, we can use the [`#get`](../models-and-databases/reference/query-set#get) method and specify the identifier value of the record we want to retrieve. For example: + +```crystal +article = Article.get(id: 1) +# => # +``` + +Now we can try to retrieve all the `Article` records that we currently have in the database. This is done by calling the `#all` method on the `Article` model: + +```crystal +Article.all +# => ]> +``` + +This method returns a `Marten::DB::Query::Set` object, which is commonly referred to as a "query set". A query set is a representation of records collections from the database that can be filtered, and iterated over. + +:::info +Please refer to [Queries](../models-and-databases/queries) to learn more about Marten's querying capabilities. +::: + +## Showing a list of articles + +Let's revisit our initial implementation of the `HomeHandler` handler we defined [earlier](#writing-a-first-handler). + +Since we are building a weblog application, it would make sense to display a list of all our `Article` objects on the index page. To do so, let's update the existing `src/handlers/home_handler.cr` file with the following content: + +```crystal title="src/handlers/home_handler.cr" +class HomeHandler < Marten::Handler + def get + render("home.html", context: { articles: Article.all }) + end +end +``` + +The `#render` method that is used above allows to return an HTTP response whose content is generated by rendering a specific [template](../templates). The template can be rendered by specifying a context hash or a named tuple. In our case the template context contains an `articles` key that maps to a query set of all the `Article` records. + +Now if you start the Marten development server again and then try to access the home page ([http://localhost:8000](http://localhost:8000)), you should get an error stating that the `home.html` template does not exist. This is normal: we need to create it. + +Templates provide a convenient way for defining the presentation logic of a web application. They allow to write HTML content that is rendered dynamically by using variables that you specify in a "template context". This rendering process can involve model records or any other variables you define. + +Let's define the expected template for our home handler by creating a `src/templates/home.html` file with the following content: + +```html title="src/templates/home.html" +{% extend "base.html" %} + +{% block content %} +

My blog

+

Articles:

+
    + {% for article in articles %} +
  • {{ article.title }}
  • + {% endfor %} +
+{% endblock %} +``` + +As you can see, Marten's templating system relies on variables that are surrounded by **`{{`** and **`}}`**. Each variable can involve lookups in order to access specific object attributes. In the above example `{{ article.title }}` means that the `title` attribute of the `article` variable should be outputted. + +Method-calling is done by using statements (also called "template tags") delimited by **`{%`** and **`%}`**. Such statements can involve for loops, if conditions, etc. In the above example we are using a for loop to iterate over the `Article` records in the `articles` query set that is "passed" to the template context in our `HomeHandler` handler. + +:::info +Please refer to [Templates](../templates/introduction) to learn more about Marten's templating system. +::: + +:::info +What about the `extend` and `block` tags in the previous snippet? These tags allow to "extend" a "base" template that usually contains the layout of an application (`base.html` in the above snippet) and to explicitly define the contents of the "blocks" that are expected by this base template. New marten projects are created with a simple `base.html` template that defines a very basic HTML document, whose body is filled with the content of a `content` block. This is why templates in this tutorial extend a `base.html` and override the content of the `content` block. + +You can learn more about these capabilities in [Template inheritance](../templates/introduction#template-inheritance). +::: + +If you go back to the home page ([http://localhost:8000](http://localhost:8000)), you should be able to see a list of article titles corresponding to all the `Article` records you created previously. + +We have now pieced together the main components of the Marten web framework (Models, Handlers, Templates). When accessing the home page of our application, the following steps are taken care of by the framework: + +1. the browser issues a GET request to `http://localhost:8000` +2. the Marten web application that is currently running receives the request +3. the Marten routing system maps the path of the incoming request to the `HomeHandler` handler +4. the handler is initialized and executed, which involves fetching all the `Article` records +5. the handler renders the `home.html` template and returns a `200 OK` response containing the rendered content +6. the Marten server sends the response with the HTML content back to the browser + +## Showing a single article + +We presently have a handler that lists all the existing `Article` records, but it would be nice to be able to actually see the content of each article individually. + +To do so, let's create a new `src/handlers/article_detail_handler.cr` handler file with the following content: + +```crystal title="src/handlers/article_detail_handler.cr" +class ArticleDetailHandler < Marten::Handler + def get + render("article_detail.html", context: { article: Article.get!(id: params["pk"]) }) + rescue Marten::DB::Errors::RecordNotFound + raise Marten::HTTP::Errors::NotFound.new("Article not found") + end +end +``` + +Like in the previous example, we will be relying on the `#render` method to render a template and return the corresponding HTTP response. But this time we will be retrieving a specific record whose parameter will be specified in a `pk` route parameter. + +:::note +You will note that we are making use of the `#get!` method to retrieve the model record in the above example. This method behaves similarly to the `#get` method we saw earlier, but it will raise a "record not found" exception if no record can be found for the specified parameters. In that case we can "rescue" this error in order to raise an "Not Found" HTTP exception that will result in a 404 response to be returned. +::: + +Let's now map this handler to a new route by adding the following line to the `config/routes.cr` file: + +```crystal title="config/routes.cr" +Marten.routes.draw do + path "/", HomeHandler, name: "home" + // highlight-next-line + path "/article/", ArticleDetailHandler, name: "article_detail" + + if Marten.env.development? + path "#{Marten.settings.assets.url}", Marten::Handlers::Defaults::Development::ServeAsset, name: "asset" + path "#{Marten.settings.media_files.url}", Marten::Handlers::Defaults::Development::ServeMediaFile, name: "media_file" + end +end +``` + +As you can see above, the new route we mapped to the `ArticleDetailHandler` handler requires a `pk` - _primary key_ - integer (`int`) parameter. Route parameters are defined using angle brackets, and the name of the parameter and its type are separated by a `:` character (`` format). + +Obviously, we also need to define the `article_detail.html` template. To do so, let's create a `src/templates/article_detail.html` with the following content: + +```html title="src/templates/article_detail.html" +{% extend "base.html" %} + +{% block content %} +

{{ article.title }}

+

{{ article.content }}

+{% endblock %} +``` + +Now if you try to access [http://localhost:8000/article/1](http://localhost:8000/article/1), you will be able to see the content of the `Article` record with ID 1. + +There is something missing though: the home page does not link to the "detail" page of each article. To remediate this, we can modify the `src/templates/home.html` template file as follows: + +```html title="src/templates/home.html" +{% extend "base.html" %} + +{% block content %} +

My blog

+

Articles:

+
    + {% for article in articles %} + // highlight-next-line +
  • + // highlight-next-line + {{ article.title }} + // highlight-next-line + ‐ View + // highlight-next-line +
  • + {% endfor %} +
+{% endblock %} +``` + +The `url` tag used in the above snippet allows to perform a reverse URL resolution. This allows to generate the final URL associated with a specific route name (the `article_detail` route name we defined earlier in this case). This reverse resolution can involve parameters if the considered route require ones. + +:::info +Please refer to [Routing](../handlers-and-http/routing) to learn more about Marten's routing system. +::: + +## Creating a new article + +So far we only implemented support for "read" operations: we made it possible to list all the available articles in the home page, and we added the ability to see the content of a specific article in the "detail" page. The next step will be to make it possible to create new articles in order to populate our weblog. + +To do so, let's start by creating a new `src/schemas/article_schema.cr` schema file with the following content: + +```crystal title="src/schemas/article_schema.cr" +class ArticleSchema < Marten::Schema + field :title, :string, max_size: 255 + field :content, :string +end +``` + +We just defined a "schema". Schemas are classes that define how input data should be serialized / deserialized, and validated. Schemas are usually used when processing web requests containing form data or pre-defined payloads. Like models, they contain a set of pre-defined fields that indicate what parameters are expected, what are their types, and how they should be validated. + +Let's see how we can use this schema in a handler. In this light, let's create a new `src/handlers/article_create_handler.cr` file: + +```crystal title="src/handlers/article_create_handler.cr" +class ArticleCreateHandler < Marten::Handler + @schema : ArticleSchema? + + def get + render("article_create.html", context: { schema: schema }) + end + + def post + if schema.valid? + article = Article.new(schema.validated_data) + article.save! + + redirect(reverse("home")) + else + render("article_create.html", context: { schema: schema }) + end + end + + private def schema + @schema ||= ArticleSchema.new(request.data) + end +end +``` + +This handler is able to handle both GET and POST requests: + +* when the incoming request is a GET, it will simply render the `article_create.html` template, and initialize the schema (instance of `ArticleSchema`) with any data currently present in the request object (which is returned by the `#request` method). This schema object is made available to the template context +* when the incoming request is a POST, it will initialize the schema and try to see if it is valid considering the incoming data. If it's valid, then the new `Article` record will be created using the schema's validated data, and the user will be redirect to the home page. Otherwise, the `article_create.html` template will be rendered again with the invalid schema in the associated context + +In the above snippet, we make use of `#redirect` in order to indicate that we want to return a 302 Found HTTP response, and we generate the redirection URL by performing a reverse resolution of the `home` route we introduced earlier by using the `#reverse` method (which is similar to the `url` template tag we encountered in the previous section). + +We can now create the `article_create.html` template file with the following content: + +```html title="src/templates/article_create.html" +{% extend "base.html" %} + +{% block content %} +

Create a new article

+
+ + +
+ + {% for error in schema.title.errors %}

{{ error.message }}

{% endfor %} + +
+ + {% for error in schema.content.errors %}

{{ error.message }}

{% endfor %} + +
+
+{% endblock %} +``` + +As you can see, the above snippet defines a form that includes two fields: one for the `title` schema field and the other one for the `content` schema field. Each schema field can be errored depending on the result of a validation, and this is why specific field errors are (optionally) displayed as well. + +:::tip What about the hidden CSRF token input? +The `csrftoken` input in the above example is mandatory because every unsafe request (eg. POST) is automatically protected by a CSRF (Cross-Site Request Forgeries) check. Please refer to [Cross-Site Request Forgery protection](../security/csrf) to learn more about this. +::: + +Finally, we need to map the `ArticleCreateHandler` handler to a proper route. We can do this by editing the `config/routes.cr` file as follows: + +```crystal title="config/routes.cr" +Marten.routes.draw do + path "/", HomeHandler, name: "home" + // highlight-next-line + path "/article/create", ArticleCreateHandler, name: "article_create" + path "/article/", ArticleDetailHandler, name: "article_detail" + + if Marten.env.development? + path "#{Marten.settings.assets.url}", Marten::Handlers::Defaults::Development::ServeAsset, name: "asset" + path "#{Marten.settings.media_files.url}", Marten::Handlers::Defaults::Development::ServeMediaFile, name: "media_file" + end +end +``` + +Now if you open your browser at [http://localhost:8000/article/create](http://localhost:8000/article/create), you should be able to see a very rough form allowing to create a new `Article` record and to redirect to it. + +Obviously, we still need to a link somewhere in our application to be able to easily access the article creation form. In this light, we can modify the `home.html` template file as follows: + +```html title="src/templates/home.html" +{% extend "base.html" %} + +{% block content %} +

My blog

+ // highlight-next-line + Create new article +

Articles:

+
    + {% for article in articles %} +
  • + {{ article.title }} + ‐ View +
  • + {% endfor %} +
+{% endblock %} +``` + +:::info +Please refer to [Schemas](../schemas/introduction) to learn more about schemas. +::: + +## Updating an article + +Now that we are able to create new `Article` records, let's add the ability to update existing records. To do so, we will implement a handler that works similarly to the `ArticleCreateHandler` handler we defined earlier: it should be able to process GET requests in order to display an "update" form, and it should validate the incoming data (and update the right record) when POST requests are submitted through an HTML form. + +In this light, let's define the `src/handlers/article_update_handler.cr` file with the following content: + +```crystal title="src/handlers/article_update_handler.cr" +class ArticleUpdateHandler < Marten::Handler + @article : Article? + @schema : ArticleSchema? + + def get + render("article_update.html", context: { article: article, schema: schema }) + end + + def post + if schema.valid? + article.update!(schema.validated_data) + redirect(reverse("home")) + else + render("article_update.html", context: { article: article, schema: schema }) + end + end + + private def article + @article ||= Article.get!(id: params["pk"]) + rescue Marten::DB::Errors::RecordNotFound + raise Marten::HTTP::Errors::NotFound.new("Article not found") + end + + private def initial_schema_data + Marten::Schema::DataHash{ "title" => article.title, "content" => article.content } + end + + private def schema + @schema ||= ArticleSchema.new(request.data, initial: initial_schema_data) + end +end +``` + +Here the `#get` and `#post` method implementations look similar to what was introduced for the `ArticleCreateHandler` handler. The main difference is that a specific `Article` record needs to be retrieved. Moreover the `ArticleSchema` schema is initialized with some "initial data" (`Marten::Schema::DataHash` hash-like object) that corresponds to the current `title` and `content` field values of the considered record. When a valid schema is processed, instead of creating a new record, we simply update the considered one through the use of the `#update!` method. + +We can now create the `article_update.html` template file with the following content: + +```html title="src/templates/article_update.html" +{% extend "base.html" %} + +{% block content %} +

Update article "{{ article.title }}"

+
+ + +
+ + {% for error in schema.title.errors %}

{{ error.message }}

{% endfor %} + +
+ + {% for error in schema.content.errors %}

{{ error.message }}

{% endfor %} + +
+
+{% endblock %} +``` + +As you can see, this looks very similar to what we did previously with the `article_create.html` template file. + +Let's now map the `ArticleUpdateHandler` handler to a proper route. We can do this by editing the `config/routes.cr` file as follows: + +```crystal title="config/routes.cr" +Marten.routes.draw do + path "/", HomeHandler, name: "home" + path "/article/create", ArticleCreateHandler, name: "article_create" + path "/article/", ArticleDetailHandler, name: "article_detail" + // highlight-next-line + path "/article//update", ArticleUpdateHandler, name: "article_update" + + if Marten.env.development? + path "#{Marten.settings.assets.url}", Marten::Handlers::Defaults::Development::ServeAsset, name: "asset" + path "#{Marten.settings.media_files.url}", Marten::Handlers::Defaults::Development::ServeMediaFile, name: "media_file" + end +end +``` + +Now if you open your browser at [http://localhost:8000/article/1/update](http://localhost:8000/article/1/update), you should be able to see a very rough form allowing to update the `Article` record with ID 1. + +We can also add a link somewhere in the home page of the application to be able to easily access the update form for existing articles. In this light, we can modify the `home.html` template file as follows: + +```html title="src/templates/home.html" +{% extend "base.html" %} + +{% block content %} +

My blog

+ Create new article +

Articles:

+
    + {% for article in articles %} +
  • + {{ article.title }} + ‐ View + // highlight-next-line + ‐ Update +
  • + {% endfor %} +
+{% endblock %} +``` + +## Deleting an article + +Finally, the last missing feature we could add is the ability to delete an article. To do so, let's introduce a handler that works as follows: when processing a GET request the handler will ask the user to confirm that they want to indeed delete the considered `Article` record, and when processing a POST request the handler will actually delete the record and then redirect to the home page of the application. + +In this light, let's define the `src/handlers/article_delete_handler.cr` file: + +```crystal title="src/handlers/article_delete_handler.cr" +class ArticleDeleteHandler < Marten::Handler + @article : Article? + + def get + render("article_delete.html", context: { article: article }) + end + + def post + article.delete + redirect(reverse("home")) + end + + private def article + @article ||= Article.get!(id: params["pk"]) + rescue Marten::DB::Errors::RecordNotFound + raise Marten::HTTP::Errors::NotFound.new("Article not found") + end +end +``` + +In the above snippet, the `#get` method simply fetches the `Article` record by using the `pk` parameter value and renders the `article_delete.html` template (with the article in the associated context). The `#post` method fetches the `Article` record as well and deletes it before redirecting the user to the home page. + +Let's now create the `article_delete.html` template file with the following content: + +```html title="src/templates/article_delete.html" +{% extend "base.html" %} + +{% block content %} +

Delete article "{{ article.title }}"

+

Are you sure?

+
+ + +
+{% endblock %} +``` + +This template simply asks the user for confirmation and displays a confirmation button embedded in a form to issue the POST request that will actually delete the record. + +Let's now map the `ArticleDeleteHandler` handler to a proper route. We can do this by editing the `config/routes.cr` file as follows: + +```crystal title="config/routes.cr" +Marten.routes.draw do + path "/", HomeHandler, name: "home" + path "/article/create", ArticleCreateHandler, name: "article_create" + path "/article/", ArticleDetailHandler, name: "article_detail" + path "/article//update", ArticleUpdateHandler, name: "article_update" + // highlight-next-line + path "/article//delete", ArticleDeleteHandler, name: "article_delete" + + if Marten.env.development? + path "#{Marten.settings.assets.url}", Marten::Handlers::Defaults::Development::ServeAsset, name: "asset" + path "#{Marten.settings.media_files.url}", Marten::Handlers::Defaults::Development::ServeMediaFile, name: "media_file" + end +end +``` + +Now if you open your browser at [http://localhost:8000/article/1/delete](http://localhost:8000/article/1/delete), you should be able to see a the confirmation page allowing to delete the `Article` record with ID 1. + +We can also add a link somewhere in the home page of the application to be able to easily access the delete confirmation page for existing articles. In this light, we can modify the `home.html` template file as follows: + +```html title="src/templates/home.html" +{% extend "base.html" %} + +{% block content %} +

My blog

+ Create new article +

Articles:

+
    + {% for article in articles %} +
  • + {{ article.title }} + ‐ View + ‐ Update + // highlight-next-line + ‐ Delete +
  • + {% endfor %} +
+{% endblock %} +``` + +## Refactoring: using template partials + +The templates used for creating and updating an article look the same: they both make use of the same schema in order to create or update articles. It would be interesting to be able to reuse this form for both templates. This is where template "partials" cames in handy: these are template snippets that can be easily "included" into other templates to avoid duplications of code. + +Let's create a `src/templates/partials/article_form.html` partial with the following content: + +```html title="src/templates/partials/article_form.html" +
+ + +
+ + {% for error in schema.title.errors %}

{{ error.message }}

{% endfor %} + +
+ + {% for error in schema.content.errors %}

{{ error.message }}

{% endfor %} + +
+
+``` + +This partial template contains the exact same form that we used in the creation and update templates. + +Let's now make use of this partial in the `src/templates/article_create.html` and `src/templates/article_update.html` templates: + +```html title="src/templates/article_create.html" +{% extend "base.html" %} + +{% block content %} +

Create a new article

+ {% include "partials/article_form.html" %} +{% endblock %} +``` + +```html title="src/templates/article_update.html" +{% extend "base.html" %} + +{% block content %} +

Update article "{{ article.title }}"

+ {% include "partials/article_form.html" %} +{% endblock %} +``` + +As you can see, the creation and update templates are now much more simple. + +## Refactoring: using generic handlers + +The handlers we implemented previously map to common web development use cases: retrieving data from the database - from a specific URL paramater - and displaying it, listing multiple objects, creating or updating records, etc. These use cases are so frequently encountered that Marten provides a set of "generic handlers" that allow to easily implement them. These generic handlers take care of these common patterns so that developers don't end up reimplementing the wheel. + +We could definitely leverage these generic handlers as part of our weblog application. + +In this light, let's start with the `HomeHandler` class we implemented earlier: this handler essentially retrieves all the `Article` records and makes this list available to the `home.html` template. This pattern is enabled by the [`Marten::Handlers::RecordList`](pathname:///api/0.3/Marten/Handlers/RecordList.html) generic handler. In order to use it, let's modify the `src/handlers/home_handler.cr` file as follows: + +```crystal title="src/handlers/home_handler.cr" +class HomeHandler < Marten::Handlers::RecordList + model Article + template_name "home.html" + list_context_name "articles" +end +``` + +In the above snippet we use a few class methods in order to define how the handler should behave: `#model` allows to define the model class that should be used to retrieve the record, `#template_name` allows to define the name of the template to render, and `#list_context_name` allows to define the name of the record list variable in the template context. + +Let's continue with the `ArticleDetailHandler` class: this handler retrieves a specific `Article` record from a `pk` route parameter, and "renders" it using a specific template. This pattern is enabled by the [`Marten::Handlers::RecordDetail`](pathname:///api/0.3/Marten/Handlers/RecordDetail.html) generic handler. In order to use it, let's modify the `src/handlers/article_detail_handler.cr` file as follows: + +```crystal title="src/handlers/article_detail_handler.cr" +class ArticleDetailHandler < Marten::Handlers::RecordDetail + model Article + template_name "article_detail.html" + record_context_name "article" +end +``` + +In order to configure how the handler should behave, we make use of a few class methods here as well: `#model` allows to define the model class of the record that should be retrieved, `#template_name` defines the template to render, and `#record_context_name` defines the name of the record variable in the template context. + +Now let's look at the `ArticleCreateHandler` class: this class displays a form when processing GET requests, and it validates a schema that is used to create a specific record when processing POST requests. This exact pattern is enabled by the [`Marten::Handlers::RecordCreate`](pathname:///api/0.3/Marten/Handlers/RecordCreate.html) generic handler. In order to use it, we can modify the `src/handlers/article_create_handler.cr` file as follows: + +```crystal title="src/handlers/article_create_handler.cr" +class ArticleCreateHandler < Marten::Handlers::RecordCreate + model Article + schema ArticleSchema + template_name "article_create.html" + success_route_name "home" +end +``` + +Here, `#model` allows to define the model class to use to create the new record, `#schema` is the schema class that should be used to validated the incoming data, `#template_name` defines the name of the template to render, and `#success_route_name` is the name of the route to redirect to after a successful record creation. + +We can now look at the `ArticleUpdateHandler` class: this class retrieves a specific record and displays a form when processing GET requests, and it validates a schema whose data is used to update the record when processing POST requests. This pattern is enabled by the [`Marten::Handlers::RecordUpdate`](pathname:///api/0.3/Marten/Handlers/RecordUpdate.html) generic handler. Let's use it and let's modify the `src/handlers/article_update_handler.cr` file as follows: + +```crystal title="src/handlers/article_update_handler.cr" +class ArticleUpdateHandler < Marten::Handlers::RecordUpdate + model Article + schema ArticleSchema + template_name "article_update.html" + success_route_name "home" + record_context_name "article" +end +``` + +Here, `#model` allows to define the model class to use to retrieve and update the record, `#schema` is the schema class that should be used to validated the incoming data, `#template_name` defines the name of the template to render, `#success_route_name` is the name of the route to redirect to after a successful record update, and `#record_context_name` is the name of the record variable in the template context. + +Finally, let's look at the `ArticleDeleteHandler` class: this handler renders a template when processing GET requests, and performs the deletion of the considered record when processing POST requests. This pattern is provided by the [`Marten::Handlers::RecordDelete`](pathname:///api/0.3/Marten/Handlers/RecordDelete.html) generic handler. In order to use it, let's modify the `src/handlers/article_delete_handler.cr` file as follows: + +```crystal title="src/handlers/article_delete_handler.cr" +class ArticleDeleteHandler < Marten::Handlers::RecordDelete + model Article + template_name "article_delete.html" + success_route_name "home" + record_context_name "article" +end +``` + +In order to configure how the handler should behave, we make use of a few class methods here as well: `#model` allows to define the model class of the record that should be retrieved and deleted, `#template_name` defines the template to render, and `#success_route_name` defines the name of the route to redirect to once the record is deleted. + +Now if you go to your application again at [http://localhost:8000](http://localhost:8000), you will notice that everything is working like it used to do before we introduced these changes (but with less code!). + +:::info +Please refer to [Generic handlers](../handlers-and-http/generic-handlers) to learn more about generic handlers. +::: + +## What's next? + +As part of this tutorial, we covered the main features of the Marten web framework by implementing a very simple application: + +* we learned to define [models](../models-and-databases) in order to interact with the database +* we learned to create [handlers](../handlers-and-http) and to map URLs to these in order to process HTTP requests +* we learned to render [templates](../templates) in order to define the presentation logic of an application + +Now that you've experimented with these core concepts of the framework, you should not hesitate to update the application we just created in order to experiment further and add new features to it. + +The Marten documentation also contains plenty of additional guides allowing you to keep exploring and learning more about other areas of the framework. These may be useful depending on the specific needs of your application: [Testing](../development/testing), [Applications](../development/applications), [Security](../security/), [Internationalization](../i18n), etc. diff --git a/docs/versioned_docs/version-0.3/handlers-and-http.mdx b/docs/versioned_docs/version-0.3/handlers-and-http.mdx new file mode 100644 index 000000000..a9be90c1e --- /dev/null +++ b/docs/versioned_docs/version-0.3/handlers-and-http.mdx @@ -0,0 +1,52 @@ +--- +title: Handlers and HTTP +--- + +import DocCard from '@theme/DocCard'; + +Handlers are classes that process a web request and return a response. They implement the necessary logic allowing to return this response, which can involve processing form data for example. + +## Guides + +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +## How-to's + +
+
+ +
+
+ +## Reference + +
+
+ +
+
+ +
+
diff --git a/docs/versioned_docs/version-0.3/handlers-and-http/cookies.md b/docs/versioned_docs/version-0.3/handlers-and-http/cookies.md new file mode 100644 index 000000000..10400918c --- /dev/null +++ b/docs/versioned_docs/version-0.3/handlers-and-http/cookies.md @@ -0,0 +1,150 @@ +--- +title: Cookies +description: Learn how to use cookies to persist data on the client. +--- + +Handlers are able to interact with a cookies store that you can use to store small amounts of data - called cookies - on the client. This data will be persisted across requests and will be made accessible with every incoming request. + +## Basic usage + +### Accessing the cookie store + +Cookies can be interacted with by leveraging a cookie store: an instance of [`Marten::HTTP::Cookies`](pathname:///api/0.3/Marten/HTTP/Cookies.html) that provides a hash-like interface allowing to retrieve and store cookie values. This cookie store can be accessed from three different places: + +* Handlers can access it through the use of the [`#cookies`](pathname:///api/0.3/Marten/Handlers/Cookies.html#cookies(*args%2C**options)-instance-method) method. +* [`Marten::HTTP::Request`](pathname:///api/0.3/Marten/HTTP/Request.html) objects give access to the cookies associated with the request via the [`#cookies`](pathname:///api/0.3/Marten/HTTP/Request.html#cookies-instance-method) method. +* [`Marten::HTTP::Response`](pathname:///api/0.3/Marten/HTTP/Response.html) objects give access to the cookies that will be returned with the HTTP response via the [`#cookies`](pathname:///api/0.3/Marten/HTTP/Response.html#cookies%3AMarten%3A%3AHTTP%3A%3ACookies-instance-method) method. + + +Here is a very simple example of how to interact with the cookies store within a handler: + +```crystal +class MyHandler < Marten::Handler + def get + cookies[:foo] = "bar" + respond "Hello World!" + end +end +``` + +### Retrieving cookie values + +The most simple way to retrieve the value of a cookie is to leverage the [`#[]`](pathname:///api/0.3/Marten/HTTP/Cookies.html#[](name%3AString|Symbol)-instance-method) method or one of its variants. + +For example, the following lines could be used to read the value of a cookie named `foo`: + +```crystal +request.cookies[:foo] # => returns the value of "foo" or raises a KeyError if not found +request.cookies[:foo]? # => returns the value of "foo" or returns nil if not found +``` + +Alternatively, the [`#fetch`](pathname:///api/0.3/Marten/HTTP/Cookies.html#fetch(name%3AString|Symbol%2Cdefault%3Dnil)-instance-method) method can also be leveraged in order to execute a block or return a default value if the specified cookie is not found: + +```crystal +request.cookies.fetch(:foo, "defaultval") +request.cookies.fetch(:foo) { "defaultval" } +``` + +### Setting cookies + +The most simple way to set a new cookie is to call the [`#[]=`](pathname:///api/0.3/Marten/HTTP/Cookies.html#[]%3D(name%2Cvalue)-instance-method) method on a cookie store. For example: + +```crystal +request.cookies[:foo] = "bar" +``` + +Calling this method will create a new cookie with the specified name and value. It should be noted that cookies created with the [`#[]=`](pathname:///api/0.3/Marten/HTTP/Cookies.html#[]%3D(name%2Cvalue)-instance-method) method will _not_ expire, will be associated with the root path (`/`), and will not be secure. + +Alternatively, it is possible to leverage the [`#set`](pathname:///api/0.3/Marten/HTTP/Cookies.html#set(name%3AString|Symbol%2Cvalue%2Cexpires%3ATime|Nil%3Dnil%2Cpath%3AString%3D"/"%2Cdomain%3AString|Nil%3Dnil%2Csecure%3ABool%3Dfalse%2Chttp_only%3ABool%3Dfalse%2Csame_site%3ANil|String|Symbol%3Dnil)%3ANil-instance-method) in order to specify custom cookie properties while setting new cookie values. For example: + +```crystal +request.cookies.set( + :foo, + "bar", + expires: 2.days.from_now, + secure: true, + same_site: "lax" +) +``` + +Appart from the cookie name and value, the [`#set`](pathname:///api/0.3/Marten/HTTP/Cookies.html#set(name%3AString|Symbol%2Cvalue%2Cexpires%3ATime|Nil%3Dnil%2Cpath%3AString%3D"/"%2Cdomain%3AString|Nil%3Dnil%2Csecure%3ABool%3Dfalse%2Chttp_only%3ABool%3Dfalse%2Csame_site%3ANil|String|Symbol%3Dnil)%3ANil-instance-method) method allows to define some additional cookie properties: + +* The cookie expiry datetime (`expires` argument). +* The cookie `path`. +* The associated `domain` (useful in order to define cross-domain cookies). +* Whether or not the cookie should be sent for HTTPS requests only (`secure` argument). +* Whether or not client-side scripts should have access to the cookie (`http_only` argument). +* The `same_site` policy (accepted values are `"lax"` or `"strict"`). + +### Deleting cookies + +Cookies can be deleted by leveraging the [`#delete`](pathname:///api/0.3/Marten/HTTP/Cookies.html#delete(name%3AString|Symbol%2Cpath%3AString%3D"/"%2Cdomain%3AString|Nil%3Dnil%2Csame_site%3ANil|String|Symbol%3Dnil)%3AString|Nil-instance-method) method. This method will delete a specific cookie and return its value, or `nil` if the cookie does not exist: + +```crystal +request.cookies.delete(:foo) +``` + +Apart from the name of the cookie, this method allows to define some additional properties of the cookie to delete: + +* The cookie `path`. +* The associated `domain` (useful in order to define cross-domain cookies). +* The `same_site` policy (accepted values are `"lax"` or `"strict"`). + +Note that the `path`, `domain`, and `same_site` values should always be the same as the ones that were used to create the cookie in the first place. Otherwise, the cookie might not be deleted properly. + +## Signed cookies + +In addition to the [regular cookie store](#accessing-the-cookie-store), Marten provides a signed cookie store version (which is accessible through the use of the [`Marten::HTTP::Cookies#signed`](pathname:///api/0.3/Marten/HTTP/Cookies.html#signed-instance-method) method) where cookies are signed (but **not** encrypted). This means that whenever a cookie is requested from this store, the signed representation of the corresponding value will be verified. This is useful to create cookies that can't be tampered by users, but it should be noted that the actual data can still be read by the client technically. + +All the methods that can be used with the regular cookie store that were highlighted in [Basic usage](#basic-usage) can also be used with the signed cookie store: + +```crystal +# Retrieving cookies... +request.signed.cookies[:foo] +request.signed.cookies[:foo]? +request.signed.cookies.fetch(:foo, "defaultval") +request.signed.cookies.fetch(:foo) { "defaultval" } + +# Setting cookies... +request.signed.cookies[:foo] = "bar" +request.signed.cookies.set(:foo, "bar", expires: 2.days.from_now) + +# Deleting cookies... +request.signed.cookies.delete(:foo) +``` + +The signed cookie store uses a [`Marten::Core::Signer`](pathname:///api/0.3/Marten/Core/Signer.html) signer object in order to sign cookie values and to verify the signature of retrieved cookies. This means that cookies are signed with HMAC signatures that use the **SHA256** hash algorithm. + +:::info +Only cookie _values_ are signed. Cookie _names_ are not signed. +::: + +## Encrypted cookies + +In addition to the [regular cookie store](#accessing-the-cookie-store), Marten provides an encrypted cookie store version (which is accessible through the use of the [`Marten::HTTP::Cookies#encrypted`](pathname:///api/0.3/Marten/HTTP/Cookies.html#encrypted-instance-method) method) where cookies are signed and encrypted. This means that whenever a cookie is requested from this store, the raw value of the cookie will be decrypted and its signature will be verified. This is useful to create cookies whose values can't be read nor tampered by users. + +All the methods that can be used with the regular cookie store that were highlighted in [Basic usage](#basic-usage) can also be used with the encrypted cookie store: + +```crystal +# Retrieving cookies... +request.encrypted.cookies[:foo] +request.encrypted.cookies[:foo]? +request.encrypted.cookies.fetch(:foo, "defaultval") +request.encrypted.cookies.fetch(:foo) { "defaultval" } + +# Setting cookies... +request.encrypted.cookies[:foo] = "bar" +request.encrypted.cookies.set(:foo, "bar", expires: 2.days.from_now) + +# Deleting cookies... +request.encrypted.cookies.delete(:foo) +``` + +The signed cookie store uses a [`Marten::Core::Encryptor`](pathname:///api/0.3/Marten/Core/Encryptor.html) encryptor object in order to encrypt and sign cookie values. This means that cookies are: + +* encrypted with an **aes-256-cbc** cipher. +* signed with HMAC signatures that use the **SHA256** hash algorithm. + +:::info +Only cookie _values_ are encrypted and signed. Cookie _names_ are not encrypted. +::: diff --git a/docs/versioned_docs/version-0.3/handlers-and-http/error-handlers.md b/docs/versioned_docs/version-0.3/handlers-and-http/error-handlers.md new file mode 100644 index 000000000..e0645f5cf --- /dev/null +++ b/docs/versioned_docs/version-0.3/handlers-and-http/error-handlers.md @@ -0,0 +1,60 @@ +--- +title: Error handlers +description: Learn about the built-in error handlers and how to configure them. +sidebar_label: Error handlers +--- + +Marten provides default error handlers that you can leverage to show specific error conditions to your users: when a page is not found, when an operation is forbidden, in case of a server error, etc. + +## The default error handlers + +Marten provides default error handlers for the following situations: + +* when **a record or a route is not found**, it will return a default Page Not Found (404) response +* when **an unexpected error occurs**, it will return a default Internal Server Error (500) response +* when **a suspicious operation is detected**, it will return a default Bad Request (400) response +* when **an action is forbidden**, it will return a default Forbidden (403) response + +Note that you don't need to manually interact with these default error handlers: they are automatically used by the Marten server when the above error conditions are met. + +### Page Not Found (404) + +A Page Not Found (404) response is automatically returned by the [`Marten::Handlers::Defaults::PageNotFound`](pathname:///api/0.3/Marten/Handlers/Defaults/PageNotFound.html) handler when: + +* a route cannot be found for an incoming request +* the [`Marten::HTTP::Errors::NotFound`](pathname:///api/0.3/Marten/HTTP/Errors/NotFound.html) exception is raised + +:::info +If your project is running in debug mode, Marten will automatically show a different page containing specific information about the original request instead of using the default Page Not Found handler. +::: + +### Internal Server Error (500) + +An Internal Server Error (500) response is automatically returned by the [`Marten::Handlers::Defaults::ServerError`](pathname:///api/0.3/Marten/Handlers/Defaults/ServerError.html) handler when an unhandled exception is intercepted by the Marten server. + +:::info +If your project is running in debug mode, Marten will automatically show a different page containing specific information about the error that occurred (traceback, request details, etc) instead of using the default Internal Server Error handler. +::: + +### Bad Request (400) + +A Bad Request (400) response is automatically returned by the [`Marten::Handlers::Defaults::BadRequest`](pathname:///api/0.3/Marten/Handlers/Defaults/BadRequest.html) handler when the [`Marten::HTTP::Errors::SuspiciousOperation`](pathname:///api/0.3/Marten/HTTP/Errors/SuspiciousOperation.html) exception is raised. + +### Forbidden (403) + +A Forbidden (403) response is automatically returned by the [`Marten::Handlers::Defaults::PermissionDenied`](pathname:///api/0.3/Marten/Handlers/Defaults/PermissionDenied.html) handler when the [`Marten::HTTP::Errors::PermissionDenied`](pathname:///api/0.3/Marten/HTTP/Errors/PermissionDenied.html) exception is raised. + +## Customizing error handlers + +Each of the error handlers mentioned above can be easily customized: by default they provide a "raw" server response with a standard message, and it might make sense on a project basis to customize how they show up to your users. As such, each handler is associated with a dedicated template name that will be rendered if your project defines it. Each of these handlers can also be replaced by a custom one by using the appropriate settings. + +These customization options are listed below: + +| Error | Template name | Handler setting | +| ----- | ------------- | ------------ | +| Page Not Found (404) | `404.html` | [`handler404`](../development/reference/settings#handler404) | +| Internal Server Error (500) | `500.html` | [`handler500`](../development/reference/settings#handler500) | +| Bad Request (400) | `400.html` | [`handler400`](../development/reference/settings#handler400) | +| Forbidden (403) | `403.html` | [`handler403`](../development/reference/settings#handler403) | + +For example, you could define a default "Page Not Found" template by defining a `404.html` HTML template file in your project's `templates` folder. diff --git a/docs/versioned_docs/version-0.3/handlers-and-http/generic-handlers.md b/docs/versioned_docs/version-0.3/handlers-and-http/generic-handlers.md new file mode 100644 index 000000000..71434c807 --- /dev/null +++ b/docs/versioned_docs/version-0.3/handlers-and-http/generic-handlers.md @@ -0,0 +1,141 @@ +--- +title: Generic handlers +description: Learn how to leverage generic handlers to perform common tasks. +sidebar_label: Generic handlers +--- + +Marten includes a set of generic handlers that can be leveraged to perform common tasks. These tasks are frequently encountered when working on web applications. For example: displaying a list of records extracted from the database, or deleting a record. Generic handlers take care of these common patterns so that developers don't end up reimplementing the wheel. + +## Scope + +Marten provides generic handlers allowing to perform the following actions: + +* redirect to a specific URL +* render an existing [template](../templates) +* process a [schema](../schemas) +* list, display, create, update, or delete [model records](../models-and-databases) + +A few of these generic handlers are described below (and all of them are listed in the [dedicated reference](./reference/generic-handlers)). Each of these handler classes must be subclassed on a per-project basis to define the required "attributes", and optionally to override methods to customize things like objects exposed in template contexts. By doing so you are essentially defining handlers that inherit these common patterns, without having to reimplement them. + +Finally, it should be noted that using generic handlers is totally optional. They provide a good starting point to implement frequently encountered use cases, but you can decide to design your own set of generic handlers to accommodate for your project needs if the built-in ones don't match your requirements. + +## A few examples + +### Performing a redirect + +Having a handler that performs a redirect can be easily achieved by subclassing the [`Marten::Handlers::Redirect`](pathname:///api/0.3/Marten/Handlers/Redirect.html) generic handler. For example, you could easily define a handler that redirects to a `articles:list` route with the following snippet: + +```crystal +class ArticlesRedirectHandler < Marten::Handlers::Redirect + route_name "articles:list" +end +``` + +The above handler will perform a reverse resolution of `articles:list` in order to get the corresponding URL and will return a 302 HTTP response (temporary redirect). + +Subclasses of this generic handler can also redirect to a plain URL and decide to return a permanent redirect (301) instead of a temporary one, for example: + +```crystal +class TestRedirectHandler < Marten::Handlers::Redirect + url "https://example.com" + permanent true +end +``` + +Finally, you can even implement your own logic in order to compute the redirection URL by overriding the `#redirect_url` method: + +```crystal +class ArticleRedirectHandler < Marten::Handlers::Redirect + def redirect_url + article = Article.get(id: params["pk"]) + if article.published? + reverse("articles:detail", pk: article.id) + else + reverse("articles:list") + end + end +end +``` + +### Rendering a template + +One of the most frequent things you will want to do when writing handlers is to return HTML responses containing rendered [templates](../templates). To do so, you can obviously define a regular handler and make use of the [`#render`](./introduction#render) helper. But, you may also want to leverage the [`Marten::Handlers::Template`](pathname:///api/0.3/Marten/Handlers/Template.html) generic handler. + +This generic handler will return a 200 OK HTTP response containing a rendered HTML template. To make use of it, you can simply define a subclass of it and ensure that you call the `#template_name` class method in order to define the template that will be rendered: + +```crystal +class HomeHandler < Marten::Handlers::Template + template_name "app/home.html" +end +``` + +If you need to, it is possible to customize the context that is used to render the configured template. To do so, you can define a `#context` method that returns a hash or a named tuple with the values you want to make available to your template: + +```crystal +class HomeHandler < Marten::Handlers::Template + template_name "app/home.html" + + def context + { "recent_articles" => Article.all.order("-published_at")[:5] } + end +end +``` + +### Displaying a model record + +It is possible to render a template that showcases a specific model record by leveraging the [`Marten::Handlers::RecordDetail`](pathname:///api/0.3/Marten/Handlers/RecordDetail.html) generic handler. + +For example, it would be possible to render an `articles/detail.html` template showcasing a specific `Article` model record with the following handler: + +```crystal +class ArticleDetailHandler < Marten::Handlers::RecordDetail + model Article + template_name "articles/detail.html" +end +``` + +By assuming that the route path associated with this handler is something like `/articles/`, this handler will automatically retrieve the right `Article` record by using the primary key provided in the `pk` route parameter. If the record does not exist, a `Marten::HTTP::Errors::NotFound` exception will be raised (which will lead to the default "not found" error page being displayed to the user), and otherwise the configured template will be rendered (with the `Article` record exposed in the context under the `record` key). + +For example, the template associated with this handler could be something like this: + +```html +
    +
  • Title: {{ record.title }}
  • +
  • Created at: {{ record.created_at }}
  • +
+``` + +### Processing a form + +It is possible to use the [`Marten::Handlers::Schema`](pathname:///api/0.3/Marten/Handlers/Schema.html) generic handler in order to process form data with a [schema](../schemas). + +To do so, it is necessary: + +* to specify the schema class to use to validate the incoming POST data through the use of the `#schema` macro +* to specify the template to render by using the `#template_name` class method: this template will likely generate an HTML form +* to specify the route to redirect to when the schema is valid via the `#success_route_name` class method + +For example: + +```crystal +class MyFormHandler < Marten::Handlers::Schema + schema MySchema + template_name "app/my_form.html" + success_route_name "home" + + def process_valid_schema + # This method is called when the schema is valid. + # You can decide to do something with the validated data... + super + end + + def process_invalid_schema + # This method is called when the schema is invalid. + super + end +end +``` + +By default, such a handler will render the configured template when the incoming request is a GET or for POST requests if the data cannot be validated using the specified schema (in that case, the template is expected to use the invalid schema to display a form with the right errored inputs). The specified template can have access to the configured schema through the use of the `schema` object in the template context. + +If the schema is valid, a temporary redirect is issued by using the URL corresponding to the `#success_route_name` value (although it should be noted that the way to generate this success URL can be overridden by defining a `#success_url` method). By default, the handler does nothing when the processed schema is valid (except redirecting to the success URL). That's why it can be helpful to override the `#process_valid_schema` method to implement any logic that should be triggered after a successful schema validation. diff --git a/docs/versioned_docs/version-0.3/handlers-and-http/how-to/create-custom-route-parameters.md b/docs/versioned_docs/version-0.3/handlers-and-http/how-to/create-custom-route-parameters.md new file mode 100644 index 000000000..d8115d2f8 --- /dev/null +++ b/docs/versioned_docs/version-0.3/handlers-and-http/how-to/create-custom-route-parameters.md @@ -0,0 +1,62 @@ +--- +title: Create custom route parameters +description: How to create custom route parameters. +--- + +Although Marten has built-in support for [common route parameters](../routing#specifying-route-parameters), it is also possible to implement your very own parameter types. This may be necessary if your routes have more complex matching requirements. + +## Defining a route parameter + +In order to implement custom parameters, you need to subclass the [`Marten::Routing::Parameter::Base`](pathname:///api/0.3/Marten/Routing/Parameter/Base.html) abstract class. Each parameter class is responsible for: + +* defining a Regex allowing to match the parameters in raw paths (which can be done by implementing a [`#regex`](pathname:///api/0.3/Marten/Routing/Parameter/Base.html#regex%3ARegex-instance-method) method) +* defining _how_ the route parameter value should be deserialized (which can be done by implementing a [`#loads`](pathname:///api/0.3/Marten/Routing/Parameter/Base.html#loads(value%3A%3A%3AString)-instance-method) method) +* defining _how_ the route parameter value should serialized (which can be done by implementing a [`#dumps`](pathname:///api/0.3/Marten/Routing/Parameter/Base.html#dumps(value)%3A%3A%3AString%3F-instance-method) method) + +The [`#regex`](pathname:///api/0.3/Marten/Routing/Parameter/Base.html#regex%3ARegex-instance-method) method takes no arguments and must return a valid [`Regex`](https://crystal-lang.org/api/Regex.html) object. + +The [`#loads`](pathname:///api/0.3/Marten/Routing/Parameter/Base.html#loads(value%3A%3A%3AString)-instance-method) method takes the raw parameter (string) as argument and is expected to return the final Crystal object corresponding to the route parameter (this is the object that will be forwarded to the handler in the route parameters hash). + +The [`#dumps`](pathname:///api/0.3/Marten/Routing/Parameter/Base.html#dumps(value)%3A%3A%3AString%3F-instance-method) method takes the final route parameter object as argument and must return the corresponding string representation. Note that this method can either return a string or `nil`: `nil` means that the passed value couldn't be serialized properly, which will make any URL reverse resolution fail with a `Marten::Routing::Errors::NoReverseMatch` error. + +For example, a "year" (1000-2999) route parameter could be implemented as follows: + +```crystal +class YearParameter < Marten::Routing::Parameter::Base + def regex : Regex + /[12][0-9]{3}/ + end + + def loads(value : ::String) : UInt64 + value.to_u64 + end + + def dumps(value) : Nil | ::String + if value.as?(UInt8 | UInt16 | UInt32 | UInt64) + value.to_s + elsif value.is_a?(Int8 | Int16 | Int32 | Int64) && [1000..2999].includes?(value) + value.to_s + else + nil + end + end +end +``` + +## Registering route parameters + +In order to be able to use custom route parameters in your [route definitions](../routing#specifying-route-parameters), you must register them to Marten's global routing parameters registry. + +To do so, you will have to call the [`Marten::Routing::Parameter#register`](pathname:///api/0.3/Marten/Routing/Parameter.html#register(id%3A%3A%3AString|Symbol%2Cparameter_klass%3ABase.class)-class-method) method with the identifier of the parameter you wish to use in route path definitions, and the actual parameter class. For example: + +```crystal +Marten::Routing::Parameter.register(:year, YearParameter) +``` + +With the above registration, you could technically create the following route definition: + +```crystal +Marten.routes.draw do + path "/vintage/", VintageHandler, name: "vintage" +end +``` diff --git a/docs/versioned_docs/version-0.3/handlers-and-http/introduction.md b/docs/versioned_docs/version-0.3/handlers-and-http/introduction.md new file mode 100644 index 000000000..4fe599656 --- /dev/null +++ b/docs/versioned_docs/version-0.3/handlers-and-http/introduction.md @@ -0,0 +1,347 @@ +--- +title: Introduction to handlers +description: Learn how to define handlers and respond to HTTP requests. +sidebar_label: Introduction +--- + +Handlers are classes whose responsibility is to process web requests and to return responses. They implement the necessary logic allowing to return this response, which can involve processing form data through the use of [schemas](../schemas) for example, retrieving [model records](../models-and-databases) from the database, etc. They can return responses corresponding to HTML pages, JSON objects, redirects, ... + +## Writing handlers + +At their core, handlers are subclasses of the [`Marten::Handler`](pathname:///api/0.3/Marten/Handlers/Base.html) class. These classes are usually defined under a `handlers` folder, at the root of a Marten project or application. Here is an example of a very simple handler: + +```crystal +class SimpleHandler < Marten::Handler + def dispatch + respond "Hello World!" + end +end +``` + +The above handler returns a `200 OK` response containing a short text, regardless of the incoming HTTP request method. + +Handlers are initialized from a [`Marten::HTTP::Request`](pathname:///api/0.3/Marten/HTTP/Request.html) object and an optional set of routing parameters. Their inner logic is executed when calling the `#dispatch` method, which _must_ return a [`Marten::HTTP::Response`](pathname:///api/0.3/Marten/HTTP/Response.html) object. + +When the `#dispatch` method is explicitly overridden, it is responsible for applying different logics in order to handle the various incoming HTTP request methods. For example, a handler might display an HTML page containing a form when handling a `GET` request, and it might process possible form data when handling a `POST` request: + +```crystal +class FormHandler < Marten::Handler + def dispatch + if request.method == 'POST' + # process form data + else + # return HTML page + end + end +end +``` + +It should be noted that this "dispatching" logic based on the incoming request method does not have to live inside an overridden `#dispatch` method. By default, each handler provides methods whose names match HTTP method verbs. This allows writing the logic allowing to process `GET` requests by overriding the `#get` method for example, or to process `POST` requests by overriding the `#post` method: + +```crystal +class FormHandler < Marten::Handler + def get + # return HTML page + end + + def post + # process form data + end +end +``` + +:::info +If a handler's logic is defined like in the above example, trying to access such handler via another HTTP verb (eg. `DELETE`) will automatically result in a "Not allowed" response (405). +::: + +### The `request` and `response` objects + +As mentioned previously, a handler is always initialized from an incoming HTTP request object (instance of [`Marten::HTTP::Request`](pathname:///api/0.3/Marten/HTTP/Request.html)) and is required to return an HTTP response object (instance of [`Marten::HTTP::Response`](pathname:///api/0.3/Marten/HTTP/Response.html)) as part of its `#dispatch` method. + +The `request` object gives access to a set of useful information and attributes associated with the incoming request. Things like the HTTP request verb, headers, or query parameters can be accessed through this object. The most common methods that you can use are listed below: + +| Method | Description | +| ----------- | ----------- | +| `#body` | Returns the raw body of the request as a string. | +| `#cookies` | Returns a hash-like object (instance of [`Marten::HTTP::Cookies`](pathname:///api/0.3/Marten/HTTP/Cookies.html)) containing the cookies associated with the request. | +| `#data` | Returns a hash-like object (instance of [`Marten::HTTP::Params::Data`](pathname:///api/0.3/Marten/HTTP/Params/Data.html)) containing the request data. | +| `#flash` | Returns a hash-like object (instance of [`Marten::HTTP::FlashStore`](pathname:///api/0.3/Marten/HTTP/FlashStore.html)) containing the flash messages available to the current request. | +| `#headers` | Returns a hash-like object (instance of [`Marten::HTTP::Headers`](pathname:///api/0.3/Marten/HTTP/Headers.html)) containing the headers embedded in the request. | +| `#host` | Returns the host associated with the considered request. | +| `#method` | Returns the considered HTTP request method (`GET`, `POST`, `PUT`, etc). | +| `#query_params` | Returns a hash-like object (instance of [`Marten::HTTP::Params::Query`](pathname:///api/0.3/Marten/HTTP/Params/Query.html)) containing the HTTP GET parameters embedded in the request. | +| `#session` | Returns a hash-like object (instance of [`Marten::HTTP::Session::Store::Base`](pathname:///api/0.3/Marten/HTTP/Session/Store/Base.html)) corresponding to the session store for the current request. | + +The `response` object corresponds to the HTTP response that is returned to the client. Response objects can be created by initializing the [`Marten::HTTP::Response`](pathname:///api/0.3/Marten/HTTP/Response.html) class directly (or one of its subclasses) or by using [response helper methods](#response-helper-methods). Once initialized, these objects can be mutated to further configure what is sent back to the browser. The most common methods that you can use in this regard are listed below: + +| Method | Description | +| ----------- | ----------- | +| `#content` | Returns the content of the response as a string. | +| `#content_type` | Returns the content type of the response as a string. | +| `#cookies` | Returns a hash-like object (instance of [`Marten::HTTP::Cookies`](pathname:///api/0.3/Marten/HTTP/Cookies.html)) containing the cookies that will be sent with the response. | +| `#headers` | Returns a hash-like object (instance of [`Marten::HTTP::Headers`](pathname:///api/0.3/Marten/HTTP/Headers.html)) containg the headers that will be used for the response. | +| `#status` | Returns the status of the response (eg. 200 or 404). | + +### Parameters + +Handlers are mapped to URLs through a [routing configuration](#mapping-handlers-to-urls). Some routes require parameters that are used by the handler to retrieve objects or perform any arbirtary logic. These parameters can be accessed by using the `#params` method, which returns a hash of all the parameters that were used to initialize the considered handler. + +For example such parameters can be used to retrieve a specific model instance: + +```crystal +class FormHandler < Marten::Handler + def get + if (record = MyModel.get(id: params["id"])) + respond "Record found: #{record}" + else + respond "Record not found!", status: 404 + end + end +end +``` + +### Response helper methods + +Technically, it is possible to forge HTTP responses by instantiating the [`Marten::HTTP::Response`](pathname:///api/0.3/Marten/HTTP/Response.html) class directly (or one of its subclasses such as [`Marten::HTTP::Response::Found`](pathname:///api/0.3/Marten/HTTP/Response/Found.html) for example). That being said, Marten provides a set of helper methods that can be used to conveniently forge responses for various use cases: + +#### `respond` + +You already saw `#respond` in action in the [first example](#writing-handlers). Basically, `#respond` allows forging an HTTP response by specifying a content, a content type, and a status code: + +```crystal +respond("Response content", content_type: "text/html", status: 200) +``` + +Unless specified, the `content_type` is set to `text/html` and the `status` is set to `200`. + +#### `render` + +`render` allows returning an HTTP response whose content is generated by rendering a specific [template](../templates). The template can be rendered by specifying a context hash or named tuple. For example: + +```crystal +render("path/to/template.html", context: { foo: "bar" }, content_type: "text/html", status: 200) +``` + +Unless specified, the `content_type` is set to `text/html` and the `status` is set to `200`. + +#### `redirect` + +`#redirect` allows forging a redirect HTTP response. It requires a `url` and accepts an optional `permanent` argument in order to define whether a permanent redirect is returned (301 Moved Permanently) or a temporary one (302 Found): + +```crystal +redirect("https://example.com", permanent: true) +``` + +Unless explicitly specified, `permanent` will automatically be set to `false`. + +#### `#head` + +`#head` allows constructing a response containing headers but without actual content. The method accepts a status code only: + +```crystal +head(404) +``` + +#### `json` + +`json` allows forging an HTTP response with the `application/json` content type. It can be used with a raw JSON string, or any serializable object: + +```crystal +json({ foo: "bar" }, status: 200) +``` + +Unless specified, the `status` is set to `200`. + +### Callbacks + +Callbacks let you define logics that are triggered before or after a handler's dispatch flow. This allows you to easily intercept the incoming request and completely bypass the execution of the regular `#dispatch` method for example. Two callbacks are supported: `before_dispatch` and `after_dispatch`. + +#### `before_dispatch` + +`before_dispatch` callbacks are executed _before_ a request is processed as part of the handler's `#dispatch` method. For example, this capability can be leveraged to inspect the incoming request and verify that a user is logged in: + +```crystal +class MyHandler < Marten::Handler + before_dispatch :require_authenticated_user + + def get + respond "Hello, authenticated user!" + end + + private def require_authenticated_user + redirect(login_url) unless user_authenticated?(request) + end +end +``` + +When one of the defined `before_dispatch` callbacks returns a [`Marten::HTTP::Response`](pathname:///api/0.3/Marten/HTTP/Response.html) object, this response is always used instead of calling the handler's `#dispatch` method (the latest is thus completely bypassed). + +#### `after_dispatch` + +`after_dispatch` callbacks are executed _after_ a request is processed as part of the handler's `#dispatch` method. For example, such a callback can be leveraged to automatically add headers or cookies to the returned response. + +```crystal +class MyHandler < Marten::Handler + after_dispatch :add_required_header + + def get + respond "Hello, authenticated user!" + end + + private def add_required_header : Nil + response!.headers["X-Foo"] = "Bar" + end +end +``` + +Similarly to `#before_dispatch` callbacks, `#after_dispatch` callbacks can return a brand new [`Marten::HTTP::Response`](pathname:///api/0.3/Marten/HTTP/Response.html) object. When this is the case, this response is always used instead of the one that was returned by the handler's `#dispatch` method. + +### Returning errors + +It is easy to forge any error response by leveraging the `#respond` or `#head` helpers that were mentioned [previously](#response-helper-methods). Using these helpers, it is possible to forge HTTP responses that are associated with specific error status codes and specific contents. For example: + +```crystal +class MyHandler < Marten::Handler + def get + respond "Content not found", status: 404 + end +end +``` + +It should be noted that Marten also support a couple of exceptions that can be raised to automatically trigger default error handlers. For example [`Marten::HTTP::Errors::NotFound`](pathname:///api/0.3/Marten/HTTP/Errors/NotFound.html) can be raised from any handler to force a 404 Not Found response to be returned. Default error handlers can be returned automatically by the framework in many situations (eg. a record is not found, or an unhandled exception is raised); you can learn more about them in [Error handlers](./error-handlers). + +## Mapping handlers to URLs + +Handlers define the logic allowing to handle incoming HTTP requests and return corresponding HTTP responses. In order to define which handler gets called for a specific URL (and what are the expected URL parameters), handlers need to be associated with a specific route. This configuration usually takes place in the `config/routes.rb` configuration file, where you can define "paths" and associate them to your handler classes: + +```crystal title="config/routes.cr" +Marten.routes.draw do + path "/", HomeHandler, name: "home" + path "/articles", ArticlesHandler, name: "articles" + path "/articles/", ArticleDetailHandler, name: "article_detail" +end +``` + +Please refer to [Routing](./routing) for more information regarding routes configuration. + +## Using cookies + +Handlers are able to interact with a cookies store, that you can use to store small amounts of data on the client. This data will be persisted across requests, and will be made accessible with every incoming request. + +The cookies store is an instance of [`Marten::HTTP::Cookies`](pathname:///api/0.3/Marten/HTTP/Cookies.html) and provides a hash-like interface allowing to retrieve and store data. Handlers can access it through the use of the `#cookies` method. Here is a very simple example of how to interact with cookies: + +```crystal +class MyHandler < Marten::Handler + def get + cookies[:foo] = "bar" + respond "Hello World!" + end +end +``` + +It should be noted that the cookies store gives access to two sub stores: an encrypted one and a signed one. + +`cookies.encrypted` allows defining cookies that will be signed and encrypted. Whenever a cookie is requested from this store, the raw value of the cookie will be decrypted. This is useful to create cookies whose values can't be read nor tampered by users: + +```crystal +cookies.encrypted[:secret_message] = "Hello!" +``` + +`cookies.signed` allows defining cookies that will be signed but not encrypted. This means that whenever a cookie is requested from this store, the signed representation of the corresponding value will be verified. This is useful to create cookies that can't be tampered by users, but it should be noted that the actual data can still be read by the client. + +```crystal +cookies.signed[:signed_message] = "Hello!" +``` + +Please refer to [Cookies](./cookies) for more information around using cookies. + +## Using sessions + +Handlers can interact with a session store, which you can use to store small amounts of data that will be persisted between requests. How much data you can persist in this store depends on the session backend being used. The default backend persists session data using an encrypted cookie. Cookies have a 4K size limit, which is usually sufficient in order to persist things like a user ID and flash messages. + +The session store is an instance of [`Marten::HTTP::Session::Store::Base`](pathname:///api/0.3/Marten/HTTP/Session/Store/Base.html) and provides a hash-like interface. Handlers can access it through the use of the `#session` method. For example: + +```crystal +class MyHandler < Marten::Handler + def get + session[:foo] = "bar" + respond "Hello World!" + end +end +``` + +Please refer to [Sessions](./sessions) for more information regarding configuring sessions and the available backends. + +## Using the flash store + +The flash store provides a way to pass basic string messages from one handler to the next one. Any string value that is set in this store will be available to the next handler processing the next request, and then it will be cleared out. Such mechanism provides a convenient way of creating one-time notification messages (such as alerts or notices). + +The flash store is an instance [`Marten::HTTP::FlashStore`](pathname:///api/0.3/Marten/HTTP/FlashStore.html) and provides a hash-like interface. Handlers can access it through the use of the `#flash` method. For example: + +```crystal +class MyHandler < Marten::Handler + def post + flash[:notice] = "Article successfully created!" + redirect("/success") + end +end +``` + +In the above example, the handler creates a flash message before returning a redirect response to another URL. It is up to the handler processing this URL to decide what to do with the flash message; this can involve rendering it as part of a base template for example. + +Note that it is possible to explicitly keep the current flash messages so that they remain all accessible to the next handler processing the next request. This can be done by using the `flash.keep` method, which can take an optional argument in order to keep the message associated with a specific key only. + +```crystal +flash.keep # keeps all the flash messages for the next request +flash.keep(:foo) # keeps the message associated with the "foo" key only +``` + +The reverse operation is also possible: you can decide to discard all the current flash messages so that none of them will remain accessible to the next handler processing the next request. This can be done by using the `flash.discard` method, which can take an optional argument in order to discard the message associated with a specific key only. + +```crystal +flash.discard # discards all the flash messages +flash.discard(:foo) # discards the message associated with the "foo" key only +``` + +## Streaming responses + +The [`Marten::HTTP::Response::Streaming`](pathname:///api/0.3/Marten/HTTP/Response/Streaming.html) response class gives you the ability to stream a response from Marten to the browser. However, unlike a standard response, this specialized class requires initialization from an [iterator](https://crystal-lang.org/api/Iterator.html) of strings instead of a content string. This approach proves to be beneficial if you intend to generate lengthy responses or responses that consume excessive memory (a classic example of this is the generation of large CSV files). + +Compared to a regular [`Marten::HTTP::Response`](pathname:///api/0.3/Marten/HTTP/Response.html) object, the [`Marten::HTTP::Response::Streaming`](pathname:///api/0.3/Marten/HTTP/Response/Streaming.html) class operates differently in two ways: + +* Instead of initializing it with a content string, it requires initialization from an [iterator](https://crystal-lang.org/api/Iterator.html) of strings. +* The response content is not directly accessible. The only way to obtain the actual response content is by iterating through the streamed content iterator, which can be accessed through the [`Marten::HTTP::Response::Streaming#streamed_content`](pathname:///api/0.3/Marten/HTTP/Response/Streaming.html#streamed_content%3AIterator(String)-instance-method) method. However, this is handled by Marten itself when sending the response to the browser, so you shouldn't need to worry about it. + +To generate streaming responses, you can either instantiate [`Marten::HTTP::Response::Streaming`](pathname:///api/0.3/Marten/HTTP/Response/Streaming.html) objects directly, or you can also leverage the [`#respond`](pathname:///api/0.3/Marten/Handlers/Base.html#respond(streamed_content%3AIterator(String)%2Ccontent_type%3DHTTP%3A%3AResponse%3A%3ADEFAULT_CONTENT_TYPE%2Cstatus%3D200)-instance-method) helper method, which works similarly to the [`#respond`](#respond) variant for response content strings. + +For example, the following handler generates a CSV and streams its content by leveraging the [`#respond`](pathname:///api/0.3/Marten/Handlers/Base.html#respond(streamed_content%3AIterator(String)%2Ccontent_type%3DHTTP%3A%3AResponse%3A%3ADEFAULT_CONTENT_TYPE%2Cstatus%3D200)-instance-method) helper method: + +```crystal +require "csv" + +class StreamingTestHandler < Marten::Handler + def get + respond(streaming_iterator, content_type: "text/csv") + end + + private def streaming_iterator + csv_io = IO::Memory.new + csv_builder = CSV::Builder.new(io: csv_io) + + (1..1000000).each.map do |idx| + csv_builder.row("Row #{idx}", "Val #{idx}") + + row_content = csv_io.to_s + + csv_io.rewind + csv_io.flush + + row_content + end + end +end +``` + +:::caution +When considering streaming responses, it is crucial to understand that the process of streaming ties up a worker process for the entire response duration. This can significantly impact your worker's performance, so it's essential to use this approach only when necessary. Generally, it's better to carry out expensive content generation tasks outside the request-response cycle to avoid any negative impact on your worker's performance. +::: diff --git a/docs/versioned_docs/version-0.3/handlers-and-http/middlewares.md b/docs/versioned_docs/version-0.3/handlers-and-http/middlewares.md new file mode 100644 index 000000000..8cb5e8913 --- /dev/null +++ b/docs/versioned_docs/version-0.3/handlers-and-http/middlewares.md @@ -0,0 +1,48 @@ +--- +title: Middlewares +description: Learn how to leverage middlewares to alter HTTP requests and responses. +sidebar_label: Middlewares +--- + + +Middlewares are used to "hook" into Marten's request/response lifecycle. They can be used to alter or implement logic based on incoming HTTP requests and the resulting HTTP responses. These hooks take an HTTP request as their input and they output an HTTP response; in the process, they can implement whatever logic they deem necessary to perform actions based on the incoming request and/or the associated response. + +## How middlewares work + +Middlewares are subclasses of the [`Marten::Middleware`](pathname:///api/0.3/Marten/Middleware.html) abstract class. They must implement a `#call` method that takes a request object (instance of [`Marten::HTTP::Request`](pathname:///api/0.3/Marten/HTTP/Request.html)) and a `get_response` proc (allowing to get the final response) as arguments, and that returns a [`Marten::HTTP::Response`](pathname:///api/0.3/Marten/HTTP/Response.html) object: + +```crystal +class TestMiddleware < Marten::Middleware + def call(request : Marten::HTTP::Request, get_response : Proc(Marten::HTTP::Response)) : Marten::HTTP::Response + # Do something with the request object. + + response = get_response.call + + # Do something with the response object. + + response + end +end +``` + +The `get_response` proc will either call the next middleware in the chain of middlewares, or the handler processing the request and returning the response. Which of these is actually called is a detail that is hidden by the `get_response` proc, and this does not matter at an individual middleware level. + +## Activating middlewares + +In order to be used, middleware classes need to be specified in the [`middleware`](../development/reference/settings#middleware) setting. This setting is an array of middleware classes that defines the "chain" of middlewares that will be "hooked" into Marten's request/response lifecycle. + +For example: + +```crystal +config.middleware = [ + Marten::Middleware::Session, + Marten::Middleware::I18n, + Marten::Middleware::GZip, +] +``` + +It should be noted that the order of middlewares is important. For example, if one of your middleware depends on a session value, you will want to ensure that it appears _after_ the `Marten::Middleware::Session` class in the `middleware` setting. + +## Available middlewares + +All the available middlewares are listed in the [dedicated reference section](./reference/middlewares). diff --git a/docs/versioned_docs/version-0.3/handlers-and-http/reference/generic-handlers.md b/docs/versioned_docs/version-0.3/handlers-and-http/reference/generic-handlers.md new file mode 100644 index 000000000..248636491 --- /dev/null +++ b/docs/versioned_docs/version-0.3/handlers-and-http/reference/generic-handlers.md @@ -0,0 +1,214 @@ +--- +title: Generic handlers +description: Generic handlers reference +--- + +This page provides a reference for all the available [generic handlers](../generic-handlers). + +## Creating a record + +**Class:** [`Marten::Handlers::RecordCreate`](pathname:///api/0.3/Marten/Handlers/RecordCreate.html) + +Handler allowing to create a new model record by processing a schema. + +This handler can be used to process a form, validate its data through the use of a [schema](../../schemas), and create a record by using the validated data. It is expected that the handler will be accessed through a GET request first: when this happens the configured template is rendered and displayed, and the configured schema which is initialized can be accessed from the template context in order to render a form for example. When the form is submitted via a POST request, the configured schema is validated using the form data. If the data is valid, the corresponding model record is created and the handler returns an HTTP redirect to a configured success URL. + +```crystal +class MyFormHandler < Marten::Handlers::RecordCreate + model MyModel + schema MyFormSchema + template_name "my_form.html" + success_route_name "my_form_success" +end +``` + +It should be noted that the redirect response issued will be a 302 (found). + +The model class used to create the new record can be configured through the use of the [`#model`](pathname:///api/0.3/Marten/Handlers/RecordCreate.html#model(model_klass)-macro) macro. The schema used to perform the validation can be defined through the use of the [`#schema`](pathname:///api/0.3/Marten/Handlers/Schema.html#schema(schema_klass)-macro) macro. Alternatively, the [`#schema_class`](pathname:///api/0.3/Marten/Handlers/Schema.html#schema_class-instance-method) method can also be overridden to dynamically define the schema class as part of the request handler handling. + +The [`#template_name`](pathname:///api/0.3/Marten/Handlers/Rendering/ClassMethods.html#template_name(template_name%3AString%3F)-instance-method) class method allows defining the name of the template to use to render the schema while the [`#success_route_name`](pathname:///api/0.3/Marten/Handlers/Schema.html#success_route_name(success_route_name%3AString%3F)-class-method) method can be used to specify the name of a route to redirect to once the schema has been validated. Alternatively, the [`#sucess_url`](pathname:///api/0.3/Marten/Handlers/Schema.html#success_url(success_url%3AString%3F)-class-method) class method can be used to provide a raw URL to redirect to. The [same method](pathname:///api/0.3/Marten/Handlers/Schema.html#success_url-instance-method) can also be overridden at the instance level to rely on a custom logic to generate the success URL to redirect to. + +## Deleting a record + +**Class:** [`Marten::Handlers::RecordDelete`](pathname:///api/0.3/Marten/Handlers/RecordDelete.html) + +Handler allowing to delete a specific model record. + +This handler can be used to delete an existing model record by issuing a POST request. Optionally the handler can be accessed with a GET request and a template can be displayed in this case; this allows to display a confirmation page to users before deleting the record: + +```crystal +class ArticleDeleteHandler < Marten::Handlers::RecordDelete + model MyModel + template_name "article_delete.html" + success_route_name "article_delete_success" +end +``` + +It should be noted that the redirect response issued will be a 302 (found). + +The [`#template_name`](pathname:///api/0.3/Marten/Handlers/Rendering/ClassMethods.html#template_name(template_name%3AString%3F)-instance-method) class method allows defining the name of the template to use to render a deletion confirmation page while the [`#success_route_name`](pathname:///api/0.3/Marten/Handlers/RecordDelete.html#success_route_name(success_route_name%3AString%3F)-class-method) method can be used to specify the name of a route to redirect to once the deletion is complete. Alternatively, the [`#sucess_url`](pathname:///api/0.3/Marten/Handlers/RecordDelete.html#success_url(success_url%3AString%3F)-class-method) class method can be used to provide a raw URL to redirect to. The [same method](pathname:///api/0.3/Marten/Handlers/RecordDelete.html#success_url-instance-method) can also be overridden at the instance level to rely on a custom logic to generate the success URL to redirect to. + +## Displaying a record + +**Class:** [`Marten::Handlers::RecordDetail`](pathname:///api/0.3/Marten/Handlers/RecordDetail.html) + +Handler allowing to display a specific model record. + +This handler can be used to retrieve a [model](../../models-and-databases/introduction) record, and to display it as part of a [rendered template](../../templates). + +```crystal +class ArticleDetailHandler < Marten::Handlers::RecordDetail + model Article + template_name "articles/detail.html" +end +``` + +The model class used to retrieve the record can be configured through the use of the [`#model`](pathname:///api/0.3/Marten/Handlers/RecordRetrieving.html#model(model_klass)-macro) macro. By default, a [`Marten::Handlers::RecordDetail`](pathname:///api/0.3/Marten/Handlers/RecordDetail.html) subclass will always retrieve model records by looking for a `pk` route parameter: this parameter is assumed to contain the value of the primary key field associated with the record that should be rendered. If you need to use a different route parameter name, you can also specify a different one through the use of the [`#lookup_param`](pathname:///api/0.3/Marten/Handlers/RecordRetrieving/ClassMethods.html#lookup_param(lookup_param%3AString|Symbol)-instance-method) class method. Finally, the model field that is used to get the model record (defaulting to `pk`) can also be configured by leveraging the [`#lookup_param`](pathname:///api/0.3/Marten/Handlers/RecordRetrieving/ClassMethods.html#lookup_param(lookup_param%3AString|Symbol)-instance-method) class method. + +The [`#template_name`](pathname:///api/0.3/Marten/Handlers/Rendering/ClassMethods.html#template_name(template_name%3AString%3F)-instance-method) class method allows defining the name of the template to use to render the considered model record. By default, the model record is associated with a `record` key in the template context, but this can also be configured by using the [`record_context_name`](pathname:///api/0.3/Marten/Handlers/RecordDetail.html#record_context_name(name%3AString|Symbol)-class-method) class method. + +## Listing records + +**Class:** [`Marten::Handlers::RecordList`](pathname:///api/0.3/Marten/Handlers/RecordList.html) + +Handler allowing to list model records. + +This base handler can be used to easily expose a list of model records: + +```crystal +class MyHandler < Marten::Handlers::RecordList + template_name "my_template" + model Post +end +``` + +The model class used to retrieve the records can be configured through the use of the [`#model`](pathname:///api/0.3/Marten/Handlers/RecordListing.html#model(model_klass)-macro) macro. The [order](../../models-and-databases/reference/query-set#order) of these model records can also be specified by leveraging the [`#ordering`](pathname:///api/0.3/Marten/Handlers/RecordListing/ClassMethods.html#page_number_param(param%3AString|Symbol)-instance-method) class method. + +The [`#template_name`](pathname:///api/0.3/Marten/Handlers/Rendering/ClassMethods.html#template_name(template_name%3AString%3F)-instance-method) class method allows defining the name of the template to use to render the list of model records. By default, the list of model records is associated with a `records` key in the template context, but this can also be configured by using the [`list_context_name`](pathname:///api/0.3/Marten/Handlers/RecordList.html#list_context_name(name%3AString|Symbol)-class-method) class method. + +Optionally, it is possible to configure that records should be [paginated](../../models-and-databases/reference/query-set#paginator) by specifying a page size through the use of the [`page_size`](pathname:///api/0.3/Marten/Handlers/RecordListing/ClassMethods.html#page_size(page_size%3AInt32%3F)-instance-method) class method: + +```crystal +class MyHandler < Marten::Handlers::RecordList + template_name "my_template" + model Post + page_size 12 +end +``` + +When records are paginated, a [`Marten::DB::Query::Page`](pathname:///api/0.3/Marten/DB/Query/Page.html) object will be exposed in the template context (instead of the raw query set). It should be noted that the page number that should be displayed is determined by looking for a `page` GET parameter by default; this parameter name can be configured as well by calling the [`page_number_param`](pathname:///api/0.3/Marten/Handlers/RecordListing/ClassMethods.html#page_number_param(param%3AString|Symbol)-instance-method) class method. + +:::tip How to customize the query set? +By default, handlers that inherit from [`Marten::Handlers::RecordList`](pathname:///api/0.3/Marten/Handlers/RecordList.html) will use a query set targetting _all_ the records of the specified model. It should be noted that you can customize this behavior easily by leveraging the [`#queryset`](pathname:///api/0.3/Marten/Handlers/RecordListing.html#queryset(queryset)-macro) macro instead of the [`#model`](pathname:///api/0.3/Marten/Handlers/RecordListing.html#model(model_klass)-macro) macro. For example: + +```crystal +class MyHandler < Marten::Handlers::RecordList + template_name "my_template" + queryset Article.filter(user: request.user) +end +``` + +Alternatively, it is also possible to override the [`#queryset`](pathname:///api/0.3/Marten/Handlers/RecordListing.html#queryset-instance-method) method and apply additional filters to the default query set: + +```crystal +class MyHandler < Marten::Handlers::RecordList + template_name "my_template" + model Article + + def queryset + super.filter(user: request.user) + end +end +``` +::: + +## Updating a record + +**Class:** [`Marten::Handlers::RecordUpdate`](pathname:///api/0.3/Marten/Handlers/RecordUpdate.html) + +Handler allowing to update a model record by processing a schema. + +This handler can be used to process a form, validate its data through the use of a [schema](../../schemas), and update an existing record by using the validated data. It is expected that the handler will be accessed through a GET request first: when this happens the configured template is rendered and displayed, and the configured schema which is initialized can be accessed from the template context to render a form for example. When the form is submitted via a POST request, the configured schema is validated using the form data. If the data is valid, the model record that was retrieved is updated and the handler returns an HTTP redirect to a configured success URL. + +```crystal +class MyFormHandler < Marten::Handlers::RecordUpdate + model MyModel + schema MyFormSchema + template_name "my_form.html" + success_route_name "my_form_success" +end +``` + +It should be noted that the redirect response issued will be a 302 (found). + +The model class used to update the new record can be configured through the use of the [`#model`](pathname:///api/0.3/Marten/Handlers/RecordRetrieving.html#model(model_klass)-macro) macro. By default, the record to update is retrieved by expecting a `pk` route parameter: this parameter is assumed to contain the value of the primary key field associated with the record that should be updated. If you need to use a different route parameter name, you can also specify a different one through the use of the [`#lookup_param`](pathname:///api/0.3/Marten/Handlers/RecordRetrieving/ClassMethods.html#lookup_param(lookup_param%3AString|Symbol)-instance-method) class method. Finally, the model field that is used to get the model record (defaulting to `pk`) can also be configured by leveraging the [`#lookup_param`](pathname:///api/0.3/Marten/Handlers/RecordRetrieving/ClassMethods.html#lookup_param(lookup_param%3AString|Symbol)-instance-method) class method. + +The schema used to perform the validation can be defined through the use of the [`#schema`](pathname:///api/0.3/Marten/Handlers/Schema.html#schema(schema_klass)-macro) macro. Alternatively, the [`#schema_class`](pathname:///api/0.3/Marten/Handlers/Schema.html#schema_class-instance-method) method can also be overridden to dynamically define the schema class as part of the request handler handling. + +The [`#template_name`](pathname:///api/0.3/Marten/Handlers/Rendering/ClassMethods.html#template_name(template_name%3AString%3F)-instance-method) class method allows defining the name of the template to use to render the schema while the [`#success_route_name`](pathname:///api/0.3/Marten/Handlers/Schema.html#success_route_name(success_route_name%3AString%3F)-class-method) method can be used to specify the name of a route to redirect to once the schema has been validated. Alternatively, the [`#sucess_url`](pathname:///api/0.3/Marten/Handlers/Schema.html#success_url(success_url%3AString%3F)-class-method) class method can be used to provide a raw URL to redirect to. The [same method](pathname:///api/0.3/Marten/Handlers/Schema.html#success_url-instance-method) can also be overridden at the instance level to rely on a custom logic to generate the success URL to redirect to. + +## Performing a redirect + +**Class:** [`Marten::Handlers::Redirect`](pathname:///api/0.3/Marten/Handlers/Redirect.html) + +Handler allowing to conveniently return redirect responses. + +This handler can be used to generate a redirect response (temporary or permanent) to another location. To configure such a location, you can either leverage the [`#route_name`](pathname:///api/0.3/Marten/Handlers/Redirect.html#route_name(route_name%3AString%3F)-class-method) class method (which expects a valid [route name](../routing#reverse-url-resolutions)) or the [`#url`](pathname:///api/0.3/Marten/Handlers/Redirect.html#url(url%3AString%3F)-class-method) class method. If you need to implement a custom redirection URL logic, you can also override the [`#redirect_url`](pathname:///api/0.3/Marten/Handlers/Redirect.html#redirect_url-instance-method) method. + +```crystal +class TestRedirectHandler < Marten::Handlers::Redirect + route_name "articles:list" +end +``` + +By default, the redirect returned by this handler is a temporary one. In order to generate a permanent redirect response instead, it is possible to leverage the [`#permanent`](pathname:///api/0.3/Marten/Handlers/Redirect.html#permanent(permanent%3ABool)-class-method) class method. + +It should also be noted that by default, incoming query string parameters **are not** forwarded to the redirection URL. If you wish to ensure that these parameters are forwarded, you can make use of the [`forward_query_string`](pathname:///api/0.3/Marten/Handlers/Redirect.html#forward_query_string(forward_query_string%3ABool)-class-method) class method. + +## Processing a schema + +**Class:** [`Marten::Handlers::Schema`](pathname:///api/0.3/Marten/Handlers/Schema.html) + +Handler allowing to process a form through the use of a [schema](../../schemas). + +This handler can be used to process a form and validate its data through the use of a [schema](../../schemas). It is expected that the handler will be accessed through a GET request first: when this happens the configured template is rendered and displayed, and the configured schema which is initialized can be accessed from the template context to render a form for example. When the form is submitted via a POST request, the configured schema is validated using the form data. If the data is valid, the handler returns an HTTP redirect to a configured success URL. + +```crystal +class MyFormHandler < Marten::Handlers::Schema + schema MyFormSchema + template_name "my_form.html" + success_route_name "my_form_success" +end +``` + +It should be noted that the redirect response issued will be a 302 (found). + +The schema used to perform the validation can be defined through the use of the [`#schema`](pathname:///api/0.3/Marten/Handlers/Schema.html#schema(schema_klass)-macro) macro. Alternatively, the [`#schema_class`](pathname:///api/0.3/Marten/Handlers/Schema.html#schema_class-instance-method) method can also be overridden to dynamically define the schema class as part of the request handler handling. + +The [`#template_name`](pathname:///api/0.3/Marten/Handlers/Rendering/ClassMethods.html#template_name(template_name%3AString%3F)-instance-method) class method allows defining the name of the template to use to render the schema while the [`#success_route_name`](pathname:///api/0.3/Marten/Handlers/Schema.html#success_route_name(success_route_name%3AString%3F)-class-method) method can be used to specify the name of a route to redirect to once the schema has been validated. Alternatively, the [`#sucess_url`](pathname:///api/0.3/Marten/Handlers/Schema.html#success_url(success_url%3AString%3F)-class-method) class method can be used to provide a raw URL to redirect to. The [same method](pathname:///api/0.3/Marten/Handlers/Schema.html#success_url-instance-method) can also be overridden at the instance level to rely on a custom logic to generate the success URL to redirect to. + +## Rendering a template + +**Class:** [`Marten::Handlers::Template`](pathname:///api/0.3/Marten/Handlers/Template.html) + +Handler allowing to respond to `GET` request with the content of a rendered HTML [template](../../templates). + +This handler can be used to render a specific template and returns the resulting content in the response. The template being rendered can be specified by leveraging the [`#template_name`](pathname:///api/0.3/Marten/Handlers/Rendering/ClassMethods.html#template_name(template_name%3AString%3F)-instance-method) class method. + +```crystal +class HomeHandler < Marten::Handlers::Template + template_name "app/home.html" +end +``` + +If you need to, it is possible to customize the context that is used to render the configured template. To do so, you can define a `#context` method that returns a hash or a named tuple with the values you want to make available to your template: + +```crystal +class HomeHandler < Marten::Handlers::Template + template_name "app/home.html" + + def context + { "recent_articles" => Article.all.order("-published_at")[:5] } + end +end +``` diff --git a/docs/versioned_docs/version-0.3/handlers-and-http/reference/middlewares.md b/docs/versioned_docs/version-0.3/handlers-and-http/reference/middlewares.md new file mode 100644 index 000000000..740634315 --- /dev/null +++ b/docs/versioned_docs/version-0.3/handlers-and-http/reference/middlewares.md @@ -0,0 +1,106 @@ +--- +title: Middlewares +description: Middlewares reference +--- + +This page provides a reference for all the available [middlewares](../middlewares). + +## Asset serving middleware + +**Class:** [`Marten::Middleware::AssetServing`](pathname:///api/0.3/Marten/Middleware/AssetServing.html) + +The purpose of this middleware is to handle the distribution of collected assets, which are stored under the configured assets root ([`assets.root`](../../development/reference/settings#root) setting). The assumption is that these assets have been "collected" using the [`collectassets`](../../development/reference/management-commands#collectassets) management command and that the file system storage ([`Marten::Core::Storage::FileSystem`](pathname:///api/0.3/Marten/Core/Storage/FileSystem.html)) is being used. + +Additionally, the [`assets.url`](../../development/reference/settings#url) setting must either align with the domain of your Marten application or correspond to a relative URL path, such as `/assets/`. This ensures proper mapping and accessibility of the assets within your application (so that they can be served by this middleware). + +It is important to mention that this middleware automatically applies compression to the served assets, utilizing GZip or deflate based on the Accept-Encoding header of the incoming request. Additionally, the middleware sets the Cache-Control header and defines a max-age of 3600 seconds, ensuring efficient caching of the assets. + +:::info +This middleware should be placed at the first position in the [`middleware`](../../development/reference/settings#middleware) setting (ie. before all other configured middlewares). +::: + +:::tip +This middleware is provided to make it easy to serve assets in situations where you can't easily configure a web server such as [Nginx](https://nginx.org) or a third-party service (like Amazon's S3 or GCS) to serve your assets directly. +::: + +## Content-Security-Policy middleware + +**Class:** [`Marten::Middleware::ContentSecurityPolicy`](pathname:///api/0.3/Marten/Middleware/ContentSecurityPolicy.html) + +This middleware guarantees the presence of the Content-Security-Policy header in the response's headers. This header provides clients with the ability to limit the allowed sources of different types of content. + +By default, the middleware will include a Content-Security-Policy header that corresponds to the policy defined in the [`content_security_policy`](../../development/reference/settings#content-security-policy-settings) settings. However, if a [`Marten::HTTP::ContentSecurityPolicy`](pathname:///api/0.3/Marten/HTTP/ContentSecurityPolicy.html) object is explicitly assigned to the request object, it will take precedence over the default policy and be used instead. + +Please refer to [Content Security Policy](../../security/content-security-policy) to learn more about the Content-Security-Policy header and how to configure it. + +## Flash middleware + +**Class:** [`Marten::Middleware::Flash`](pathname:///api/0.3/Marten/Middleware/Flash.html) + +Enables the use of [flash messages](../introduction#using-the-flash-store). + +When this middleware is used, each request will have a flash store initialized and populated from the request's session store. This flash store is a hash-like object that allows to fetch or set values that are associated with specific keys, and that will only be available to the next request (after that they are cleared out). + +The flash store depends on the presence of a working session store. As such, the [Session middleware](#session-middleware) MUST be used along with this middleware. Moreover, this middleware must be placed _after_ the [`Marten::Middleware::Session`](pathname:///api/0.3/Marten/Middleware/Session.html) in the [`middleware`](../../development/reference/settings#middleware) setting. + +## GZip middleware + +**Class:** [`Marten::Middleware::GZip`](pathname:///api/0.3/Marten/Middleware/GZip.html) + +Compresses the content of the response if the browser supports GZip compression. + +This middleware will compress responses that are big enough (200 bytes or more) if they don't already contain an Accept-Encoding header. It will also set the Vary header correctly by including Accept-Encoding in it so that caches take into account the fact that the content can be compressed or not. + +The GZip middleware should be positioned before any other middleware that needs to interact with the response content in the [`middleware`](../../development/reference/settings#middleware) setting. This is to ensure that the compression happens only when the response content is no longer accessed. + +## I18n middleware + +**Class:** [`Marten::Middleware::I18n`](pathname:///api/0.3/Marten/Middleware/I18n.html) + +Activates the right I18n locale based on incoming requests. + +This middleware will activate the right locale based on the Accept-Language header or the value provided by the [locale cookie](../../development/reference/settings#locale_cookie_name). Only explicitly-configured locales can be activated by this middleware (that is, locales that are specified in the [`i18n.available_locales`](../../development/reference/settings#available_locales) and [`i18n.default_locale`](../../development/reference/settings#default_locale) settings). If the incoming locale can't be found in the project configuration, the default locale will be used instead. + +## Session middleware + +**Class:** [`Marten::Middleware::Session`](pathname:///api/0.3/Marten/Middleware/Session.html) + +Enables the use of [sessions](../sessions). + +When this middleware is used, each request will have a session store initialized according to the [sessions configuration](../../development/reference/settings#sessions-settings). This session store is a hash-like object that allows to fetch or set values that are associated with specific keys. + +The session store is initialized from a session key that is stored as a regular cookie. If the session store ends up being empty after a request's handling, the associated cookie is deleted. Otherwise, the cookie is refreshed if the session store is modified as part of the considered request. Each session cookie is set to expire according to a configured cookie max age (the default cookie max age is 2 weeks). + +## SSL redirect middleware + +**Class:** [`Marten::Middleware::SSLRedirect`](pathname:///api/0.3/Marten/Middleware/SSLRedirect.html) + +Redirects all non-HTTPS requests to HTTPS. + +This middleware will permanently redirect all non-HTTP requests to HTTPS. By default the middleware will redirect to the incoming request's host, but a different host to redirect to can be configured with the [`ssl_redirect.host`](../../development/reference/settings#host-2) setting. Additionally, specific request paths can also be exempted from this SSL redirect if the corresponding strings or regexes are specified in the [`ssl_redirect.exempted_paths`](../../development/reference/settings#exempted_paths) setting. + +## Strict-Transport-Security middleware + +**Class:** [`Marten::Middleware::StrictTransportSecurity`](pathname:///api/0.3/Marten/Middleware/StrictTransportSecurity.html) + +Sets the Strict-Transport-Security header in the response if it wasn't already set. + +This middleware automatically sets the HTTP Strict-Transport-Security (HSTS) response header for all responses unless it was already specified in the response headers. This allows to let browsers know that the considered website should only be accessed using HTTPS, which results in future HTTP requests being automatically converted to HTTPS (up until the configured strict transport policy max age is reached). + +Browsers ensure that this policy is applied for a specific duration because a `max-age` directive is embedded into the header value. This max age duration is expressed in seconds and can be configured using the [`strict_security_policy.max_age`](../../development/reference/settings#max_age) setting. + +:::caution +When enabling this middleware, you should probably start with small values for the [`strict_security_policy.max_age`](../../development/reference/settings#max_age) setting (for example `3600` - one hour). Indeed, when browsers are aware of the Strict-Transport-Security header they will refuse to connect to your website using HTTP until the expiry time corresponding to the configured max age is reached. + +This is why the value of the [`strict_security_policy.max_age`](../../development/reference/settings#max_age) setting is `nil` by default: this prevents the middleware from inserting the Strict-Transport-Security response header until you actually specify a max age. +::: + +## X-Frame-Options middleware + +**Class:** [`Marten::Middleware::XFrameOptions`](pathname:///api/0.3/Marten/Middleware/XFrameOptions.html) + +Sets the X-Frame-Options header in the response if it wasn't already set. + +When this middleware is used, a X-Frame-Options header will be inserted into the HTTP response. The default value for this header (which is configurable via the [`x_frame_options`](../../development/reference/settings#x_frame_options) setting) is "DENY", which means that the response cannot be displayed in a frame. This allows preventing click-jacking attacks, by ensuring that the web app cannot be embedded into other sites. + +On the other hand, if the `x_frame_options` is set to "SAMEORIGIN", the page can be displayed in a frame if the including site is the same as the one serving the page. diff --git a/docs/versioned_docs/version-0.3/handlers-and-http/routing.md b/docs/versioned_docs/version-0.3/handlers-and-http/routing.md new file mode 100644 index 000000000..e353d59b2 --- /dev/null +++ b/docs/versioned_docs/version-0.3/handlers-and-http/routing.md @@ -0,0 +1,144 @@ +--- +title: Routing +description: Learn how to map handlers to routes. +sidebar_label: Routing +--- + +Marten gives you the ability to design your URLs the way you want, by allowing you to easily map routes to specific [handlers](./introduction), and by letting you generate paths and URLs from your application code. + +## The basics + +In order to access a handler via a browser, it is necessary to map it to a URL route. To do so, route "maps" can be used to define mappings between route paths and existing handler classes. These route maps can be as long or as short as needed, but it is generally a good idea to create sub routes maps that are included in the main routes map. + +The main routes map usually lives in the `config/routes.cr` file. For example, the content of such a file could look like this: + +```crystal +Marten.routes.draw do + path "/", HomeHandler, name: "home" + path "/articles", ArticlesHandler, name: "articles" + path "/articles/", ArticleDetailHandler, name: "article_detail" +end +``` + +As you can see, routes are defined by calling a `#path` method that requires three arguments: + +* the first argument is the route pattern, which is a string like `/foo/bar` and which can contain additional [parameters](#specifying-route-parameters) +* the second argument is the handler class associated with the specified route +* the last argument is the route name, which is an identifier that can later be used in your codebase to generate the full URL for a specific route, and optionally inject parameters in it (see [Reverse URL resolutions](#reverse-url-resolutions)) + +:::tip +It is possible to map multiple routes to the same handler class if necessary. This can be useful if you need to provide route aliases for some handlers for example. +::: + +These routes are evaluated and constructed at runtime, which means that you can define conditional routes if you need to. For example, a "debug" handler (only available in a development environment) could be added to the above routes map with the following addition: + +```crystal +Marten.routes.draw do + path "/", HomeHandler, name: "home" + path "/articles", ArticlesHandler, name: "articles" + path "/articles/", ArticleDetailHandler, name: "article_detail" + + // highlight-next-line + if Marten.env.development? + // highlight-next-line + path "/debug", DebugHandler, name: "debug" + // highlight-next-line + end +end +``` + +When a URL is requested, Marten runs through all the defined routes to identify a matching one. The handler associated with this route will be initialized from the route parameters (if there are any) and the handler object will be used to respond to the considered request. + +It should be noted that if no route is matched for a specific URL, Marten will automatically return a 404 Not Found response, by leveraging a configurable [error handler](./error-handlers). + +## Specifying route parameters + +As highlighted in the previous examples, route parameters can be defined using angle brackets. Each route parameter must define a mandatory name and an optional type using the following syntaxes: + +* `` +* `` + +When no type is specified for a parameter, any string excluding the forward slash character (**`/`**) will be matched. + +The following route parameter types are available: + +| Type | Description | +| ----------- | ----------- | +| `str` or `string` | Matches any non-empty string (excluding the forward slash character **`/`**). This is the default parameter type used for untyped parameters (eg. ``). | +| `int` | Matches zero or any positive integer. These parameter values are always deserialized as `UInt64` objects. | +| `path` | Matches any non-empty strings including forward slash characters (**`/`**). For example `foo/bar/xyz` could be matched by this parameter type. | +| `slug` | Matches any string containing only ASCII letters, numbers, hyphen, and underscore characters. For example `my-first-project-01` could be matched by this parameter type. | +| `uuid` | Matches a valid UUID string. These parameter values are always deserialized as `UUID` objects. | + +It should be noted that it is possible to register custom route parameter implementations if needed. See [Create custom route parameters](./how-to/create-custom-route-parameters) to learn more about this capability. + +## Defining included routes + +The main routes map (which usually lives in the `config/routes.cr` file) does not have to be a "flat" definition of all the available routes. Indeed, you can define "sub" routes maps if you need to and "include" these in your main routes map. + +This capability can be extremely useful to "include" a set of routes from an installed application (a third-party library or one of your in-project applications). This also allows better organizing route namespaces and bundling a set of related routes under a similar prefix. + +For example, a main routes map and an article routes map could be defined as follows: + +```crystal +ARTICLE_ROUTES = Marten::Routing::Map.draw do + path "", ArticlesHandler, name: "list" + path "/create", ArticleCreateHandler, name: "create" + path "/", ArticleDetailHandler, name: "detail" + path "//update", ArticleUpdateHandler, name: "update" + path "//delete", ArticleDeleteHandler, name: "delete" +end + +Marten.routes.draw do + path "/", HomeHandler, name: "home" + path "/articles", Articles::ROUTES, name: "articles" +end +``` + +In the above example, the following URLs would be generated by Marten in addition to the root URL: + +| URL | Handler | Name | +| --- | ---- | ---- | +| `/articles` | `ArticlesHandler` | `articles:list` | +| `/articles/create` | `ArticleCreateHandler` | `articles:create` | +| `/articles/` | `ArticleDetailHandler` | `articles:detail` | +| `/articles//update` | `ArticleUpdateHandler` | `articles:update` | +| `/articles//delete` | `ArticleDeleteHandler` | `articles:delete` | + +As you can see, both the URLs and the route names end up being prefixed respectively with the path and the name specified in the including route. + +Note that the sub routes map does not have to live in the `config/routes.cr` file: it can technically live anywhere in your codebase. The ideal way to define the routes map of a specific application would be to put it in a `routes.cr` file in the application's directory. + +When Marten encounters a path that leads to another sub routes map, it chops off the part of the URL that was matched up to that point and then forwards the remaining to the sub routes map in order to see if it is matched by one of the underlying routes. + +## Reverse URL resolutions + +When working with web applications, a frequent need is to generate URLs in their final forms. To do so, you will want to avoid hard-coding URLs and instead leverage the ability to generate them from their associated names: this is what we call a reverse URL resolution. + +"Reversing" a URL is as simple as calling the [`Marten::Routing::Map#reverse`](pathname:///api/0.3/Marten/Routing/Map.html#reverse(name%3AString|Symbol%2Cparams%3AHash(String|Symbol%2CParameter%3A%3ATypes))-instance-method) method from the main routes map, which is accessible through the use of the [`Marten#routes`](pathname:///api/0.3/Marten.html#routes-class-method) method: + +```crystal +Marten.routes.reverse("home") # will return "/" +``` + +In order to reverse a URL from within a handler class, you can simply leverage the [`Marten::Handlers::Base#reverse`](pathname:///api/0.3/Marten/Handlers/Base.html#reverse(*args%2C**options)-instance-method) handler method: + +```crystal +class MyHandler < Marten::Handler + def post + redirect(reverse("home")) + end +end +``` + +As highlighted previously, some routes require one or more parameters and in order to reverse these URLs you can simply specify these parameters as arguments when calling `#reverse`: + +```crystal +Marten.routes.reverse("article_detail", pk: 42) # will return "/articles/42" +``` + +Finally, it should be noted that the namespaces that are created when defining [included routes](#defining-included-routes) also apply when reversing the corresponding URLs. For example, the name allowing to reverse the URL associated with the `ArticleUpdateHandler` in the previous snippet would be `articles:update`: + +```crystal +Marten.routes.reverse("articles:update", pk: 42) # will return "/articles/42/update" +``` diff --git a/docs/versioned_docs/version-0.3/handlers-and-http/sessions.md b/docs/versioned_docs/version-0.3/handlers-and-http/sessions.md new file mode 100644 index 000000000..32a7f927a --- /dev/null +++ b/docs/versioned_docs/version-0.3/handlers-and-http/sessions.md @@ -0,0 +1,62 @@ +--- +title: Sessions +description: Learn how to use sessions to persist data between requests. +sidebar_label: Sessions +--- + +Sessions can be used to store small amounts of data that will be persisted between requests, on a per-visitor basis. This data is usually stored on the backend side (depending on the chosen session storage), and it is associated with a session ID that is persisted on the client through the use of a dedicated cookie. + +## Configuration + +In order to use sessions, you need to make sure that the [`Marten::Middleware::Session`](pathname:///api/0.3/Marten/Middleware/Session.html) middleware is part of your project's middleware chain, which can be configured in the [`middleware`](../development/reference/settings#middleware) setting. Note that the session middleware class is automatically added to this setting when initializing new projects. + +If your project does not require the use of sessions, you can simply ensure that the [`middleware`](../development/reference/settings#middleware) setting does not include the [`Marten::Middleware::Session`](pathname:///api/0.3/Marten/Middleware/Session.html) middleware class. + +How the session ID cookie is generated can also be tweaked by leveraging the following settings: + +* [`sessions.cookie_domain`](../development/reference/settings#cookie_domain-1) +* [`sessions.cookie_http_only`](../development/reference/settings#cookie_http_only-1) +* [`sessions.cookie_max_age`](../development/reference/settings#cookie_max_age-1) +* [`sessions.cookie_name`](../development/reference/settings#cookie_name-1) +* [`sessions.cookie_same_site`](../development/reference/settings#cookie_same_site-1) +* [`sessions.cookie_secure`](../development/reference/settings#cookie_secure-1) + +## Session stores + +How session data is actually persisted can be defined by configuring the right session store backend, which can be done through the use of the [`sessions.store`](../development/reference/settings#store) setting. + +By default, sessions are stored within a single cookie (`:cookie` session store). Cookies have a 4K size limit, which is usually sufficient in order to persist things like a user ID and flash messages. `:cookie` is the only store that is built in the Marten web framework presently. + +Other session stores can be installed as separate shards. For example, the [`marten-db-session`](https://github.com/martenframework/marten-db-session) shard can be leveraged to persist session data in the database. + +## Using sessions + +When the [`Marten::Middleware::Session`](pathname:///api/0.3/Marten/Middleware/Session.html) middleware is used, each HTTP request object will have a [`#session`](pathname:///api/0.3//Marten/HTTP/Request.html#session-instance-method) method returning the session store for the current request. The session store is an instance of [`Marten::HTTP::Session::Store::Base`](pathname:///api/0.3/Marten/HTTP/Session/Store/Base.html) and provides a hash-like interface: + +```crystal +# Persisting values: +request.session[:foo] = "bar" + +# Accessing values: +request.session[:foo] +request.session[:foo]? + +# Deleting values: +request.session.delete(:foo) + +# Checking emptiness: +request.session.empty? +``` + +Both symbol and string keys can be used when trying to interact with the session store, but only **string values** can be stored. + +If you are trying to access the session store from within a handler, it should be noted that you can leverage the `#session` method instead of using the request object: + +```crystal +class MyHandler < Marten::Handler + def get + session[:foo] = "bar" + respond "Hello World!" + end +end +``` diff --git a/docs/versioned_docs/version-0.3/i18n.mdx b/docs/versioned_docs/version-0.3/i18n.mdx new file mode 100644 index 000000000..c862f64eb --- /dev/null +++ b/docs/versioned_docs/version-0.3/i18n.mdx @@ -0,0 +1,15 @@ +--- +title: Internationalization +--- + +import DocCard from '@theme/DocCard'; + +Marten provides various tools and mechanisms that you can leverage to benefit from translations and localized contents in your projects. + +## Guides + +
+
+ +
+
diff --git a/docs/versioned_docs/version-0.3/i18n/introduction.md b/docs/versioned_docs/version-0.3/i18n/introduction.md new file mode 100644 index 000000000..85ecf30de --- /dev/null +++ b/docs/versioned_docs/version-0.3/i18n/introduction.md @@ -0,0 +1,161 @@ +--- +title: Introduction to internationalization +description: Learn how to leverage translations and localized contents in your Marten projects. +sidebar_label: Introduction +--- + +Marten provides integration with [crystal-i18n](https://crystal-i18n.github.io/) to make it possible to leverage translations and localized content in your Marten projects. + +## Overview + +Internationalization and localization are techniques allowing a website to provide content using languages and formats that are adapted to specific audiences. + +Marten's internationalization and localization integration rely on the use of the [crystal-i18n](https://crystal-i18n.github.io/) shard, which provides a unified interface allowing to leverage translations and localized contents in a Crystal project. You don't have to manually install this shard in your projects: it is a dependency of the framework itself, and as such, it is automatically installed with Marten. + +Crystal-I18n makes it easy to configure translations and formats for a specific set of locales. These can be used to perform translation lookups and localization. With this library, translations can be defined through the use of dedicated "loaders" (abstractions that load the translations from a specific source and make them available to the I18n API). For example, translations can be loaded from a YAML file, a JSON file, or something entirely different if needed. + +Marten itself defines a set of translated contents for things that should be internationalized (eg. model field errors or schema field errors) that are loaded through the use of the regular YAML loader. Other [app-specific translations](#locales-and-apps) must also be defined as YAML files since they are loaded using a YAML loader as well. + +## Configuration + +Marten provides an integration allowing to configure internationalization-related settings. These settings are available under the [`i18n`](../development/reference/settings#i18n-settings) namespace and allow to define things like the default locale and the available locales: + +```crystal +config.i18n.default_locale = :fr +config.i18n.available_locales = [:en, :fr] +``` + +You can also leverage the [various configuration options](https://crystal-i18n.github.io/configuration.html) that are provided by this shard to further configure how translations should be performed. By doing so you can add more custom I18n backend loaders for example. + +:::tip +If you need to further [configure Crystal I18n](https://crystal-i18n.github.io/configuration.html), you should probably define a dedicated initializer file under the `config/initializers` folder. +::: + +## Basic usage + +As stated before, Marten relies on the [crystal-i18n](https://crystal-i18n.github.io/) shard, which means that you can also look at the dedicated documentation to learn more about this shard and its configuration options. The following section mainly highlights some of the main features of this library. + +### Defining translations + +Translations must be defined in a `locales` folder at the root of an application. For example, if you are using the [main application](../development/applications#the-main-application) (which corresponds to the standard `src` folder) you could define a `src/locales` folder containing an `en.yml` file as follows: + +``` +myproject/ +├── src +│   ├── locales +│   │   ├── en.yml +``` + +Translations inside a YAML file must be namespaced to the locale they are associated with (`en` in this case). Example content for our `en.yml` file could look like this: + +```yaml title=src/en.yml +en: + message: "This is a message" + simple: + translation: "This is a simple translation" + interpolation: "Hello, %{name}!" +``` + +The "path" leading to a translation in such files is important because it corresponds to the key that should be used when performing [translation lookups](#translations-lookups). For example, `simple.translation` would be the key to use in order to translate the corresponding message. + +It should be noted that the `%{var}` syntax in the above example is used to define _interpolations_: these variables must be specified when performing translation lookups so that their values are inserted in the translated strings. + +### Translations lookups + +Translation lookups can be performed by leveraging the `I18n#translate` or `I18n#translate!` methods. Those methods try to find a matching translation for a specific key, which can be comprised of multiple namespaces or scopes separated by a dot (.): this key corresponds to the "path" leading to the actual translation (as mentioned before). + +The `I18n#translate` and `I18n#translate!` methods differ in regards to how they handle missing translations: + +* `I18n#translate` returns a message indicating that the translation is missing +* `I18n#translate!` raises a specific exception + +For example, given the translations defined in [Defining translations](#defining-translations), we could perform the following translation lookups: + +```crystal +I18n.translate(:message) # => "This is a message" +I18n.translate("simple.translation") # => "This is a simple translation" +I18n.translate("simple.interpolation", name: "John Doe") # => "Hello, John Doe!" +``` + +This only scratches the surface of what's possible in terms of translation lookups. You can refer to the [dedicated documentation](https://crystal-i18n.github.io/translation_lookups.html), and more specifically the [interpolations](https://crystal-i18n.github.io/translation_lookups.html#interpolations) and [pluralizations](https://crystal-i18n.github.io/translation_lookups.html#pluralization) sections, to learn about these capabilities. + +### Localization + +Localization of datetimes and numbers can be achieved through the use of the `I18n#localize` method. In both cases, localization _formats_ need to be defined in your locale files. There are a lot of available formats at your disposal (and all of them are documented in the [related documentation](https://crystal-i18n.github.io/localization.html)). For example, the following translations could be used to format dates in English: + +```yaml +en: + i18n: + date: + month_names: [January, February, March, April, May, June, + July, August, September, October, November, December] + formats: + default: "%Y-%m-%d" + long: "%B %d, %Y" +``` + +The above structure is expected by Crystal I18n and defines basic translations for the relevant directives that can be outputted when localizing dates. It also defines a few formats under the `i18n.date.formats` scope: among these formats, only the default one is really mandatory since this is the one that is used by default if no other format is explicitly provided to the `I18n#localize` method. All these formats make use of the directives defined by the [`Time::Format`](https://crystal-lang.org/api/Time/Format.html) struct. + +Given the above translations, you could localize date objects as follows: + +```crystal +I18n.localize(Time.local.date) # outputs "2020-12-13" +I18n.localize(Time.local.date, :long) # outputs "December 13, 2020" +``` + +### Switching locales + +Once you have defined translations, it is generally needed to explicitly "activate" the use of a specific locale in order to ensure that the right translations are generated for your users. In this light, the current locale can be specified using the `I18n#activate` method: + +```crystal +I18n.activate(:fr) +``` + +When activating a locale with `I18n#activate`, all further translations or localizations will be done using the specified locale. + +Note that it is also possible to execute a block with a specific locale activated. This can be done by using the `I18n#with_locale` method: + +```crystal +I18n.with_locale(:fr) do + I18n.t("simple.translation") # Will output a text in french +end +``` + +Finally, it should be noted that Marten provides an [I18n middleware](../handlers-and-http/reference/middlewares#i18n-middleware) that activates the right locale based on the Accept-Language header. Only explicitly-configured locales can be activated by this middleware (that is, locales that are specified in the [`i18n.available_locales`](../development/reference/settings#available_locales) and [`i18n.default_locale`](../development/reference/settings#default_locale) settings). If the incoming locale can't be found in the project configuration, the default locale will be used instead. By leveraging this middleware, you can be sure that the right locale is automatically enabled for your users, so you don't need to take care of it. + +## Locales and apps + +As mentioned previously, each [application](../development/applications) can define translations inside a `locales` folder that must be located at the root of the application's directory. This `locales` folder should contain YAML files defining the translations that are required by the application. + +The way to organize translations inside this folder is left to application developers. That being said, it is necessary to ensure that all the YAML files containing translations are namespaced with the targeted locale (eg. `en`, `fr`, etc). + +Moreover, it is also recommended to explicitly namespace an application's translations by using an identifier that is unique for the considered application. For example, a `foo` application could define a `message` translation and another `bar` application could define a `message` translation as well. If these translation keys are not properly namespaced, one of the translations will be overridden by the one of the other application. The best way to avoid this is to namespace all the translations of an application with the identifier of the application itself. For example: + +```yaml +en: + foo: + message: This is a message +``` + +In this case, the `foo` application's codebase would request translations using the `foo.message` key, which makes it impossible to encounter conflict issues with other application translations. + +## How Marten resolves the current locale + +Marten will attempt to determine the "current" locale for activation only when the [I18n middleware](../handlers-and-http/reference/middlewares#i18n-middleware) is used. + +This middleware can activate the appropriate locale by considering the following: + +* The value of the Accept-Language header. +* The value of a cookie, with its name defined by the [`i18n.locale_cookie_name`](../development/reference/settings#locale_cookie_name) setting. + + +The [I18n middleware](../handlers-and-http/reference/middlewares#i18n-middleware) only allows activation of explicitly configured locales, which are specified in the [`i18n.available_locales`](../development/reference/settings#available_locales) and [`i18n.default_locale`](../development/reference/settings#default_locale) settings. If the incoming locale is not found in the project configuration, the default locale will be used instead. By utilizing this middleware, you can be sure that the right locale is automatically enabled for your users, so that you don't need to take care of it. + +## Limitations + +It's important to be aware of a few limitations when working with translations powered by [Crystal I18n](https://crystal-i18n.github.io/) within a Marten project: + +1. Marten automatically configures YAML translation loaders for applications, and it is not currently possible to use other loader types (such as JSON) presently +2. Marten does not allow the use of "embedded" translations for applications since those are discovered and configured at runtime: as such application translations are treated as "assets" that must be deployed along with the compiled binary + +Note that these restrictions do not prevent the use of custom translation backends if necessary. Please refer to the [related documentation](https://crystal-i18n.github.io/configuration.html#loaders) if you need to use custom translation loaders in your projects. diff --git a/docs/versioned_docs/version-0.3/models-and-databases.mdx b/docs/versioned_docs/version-0.3/models-and-databases.mdx new file mode 100644 index 000000000..dd77374f0 --- /dev/null +++ b/docs/versioned_docs/version-0.3/models-and-databases.mdx @@ -0,0 +1,61 @@ +--- +title: Models and databases +--- + +import DocCard from '@theme/DocCard'; + +Models define what data can be persisted and manipulated by a Marten application. They explicitly specify fields and rules that map to database tables and columns. + +## Guides + +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +
+
+ +## How-to's + +
+
+ +
+
+ +## Reference + +
+
+ +
+
+ +
+
+ +
+
+ +
+
diff --git a/docs/versioned_docs/version-0.3/models-and-databases/callbacks.md b/docs/versioned_docs/version-0.3/models-and-databases/callbacks.md new file mode 100644 index 000000000..67ebeb9f5 --- /dev/null +++ b/docs/versioned_docs/version-0.3/models-and-databases/callbacks.md @@ -0,0 +1,114 @@ +--- +title: Model callbacks +description: Learn how to define model callbacks. +sidebar_label: Callbacks +--- + +Models callbacks let you define logic that is triggered before or after a record's state alteration. They are methods that get called at specific stages of a record's lifecycle. For example, callbacks can be called when model instances are created, updated, or deleted. + +This documents covers the available callbacks and introduces you to the associated API, which you can use to define hooks in your models. + +## Overview + +As stated above, callbacks are methods that will be called when specific events occur for a specific model instance. They need to be registered explicitly as part your model definitions. There are [many types of of callbacks](#available-callbacks), and it is possible to register "before" or "after" callbacks for most of these types. + +Registering a callback is as simple as calling the right callback macro (eg. `#before_validation`) with a symbol of the name of the method to call when the callback is executed. For example: + +```crystal +class User < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :username, :string, max_size: 64, unique: true + + before_validation :ensure_username_is_downcased + + private def ensure_username_is_downcased + self.username = username.try(&.downcase) + end +end +``` + +In the above snippet, a `before_validation` callback is registered to ensure that the `username` of a `User` instance is downcased before any validation. + +This technique of callback registration is shared by all types of callbacks. + +It should be noted that the order in which callback methods are registered for a given callback type (eg. `before_update`) matters: callbacks will be called in the order in which they were registered. + +## Available callbacks + +### `after_initialize` + +`after_initialize` callbacks are called right after a model instance is initialized. They will be called automatically when new model instances are initialized through the use of `new` or when records are retrieved from the database. + +### `before_validation` and `after_validation` + +`before_validation` callbacks are called before running validation rules for a given model instance while `after_validation` callbacks are executed after. They can be used to sanitize model instance attributes for example. + +The use of methods like `#valid?` or `#invalid?`, or any other methods involving validations (`#save`, `#save!`, `#create`, or `#create!`), will trigger validation callbacks. See [Model validations](./validations) for more details. + +### `before_create` and `after_create` + +`before_create` callbacks are called before a new record is inserted into the database. `after_create` callbacks are called after a new record has been created at the database level. + +The use of the `#save` method (or `#save!`) on a new model instance will trigger the execution of creation callbacks. The use of the `#create` / `#create!` methods will also trigger these callbacks. + +### `before_update` and `after_update` + +`before_update` callbacks are called before an existing record is updated while `after_update` callbacks are called after. + +The use of the `#save` method (or `#save!`) on an existing model record will trigger the execution of update callbacks. + +### `before_save` and `after_save` + +`before_save` callbacks are called before a record (existing or new) is saved to the database while `after_save` callbacks are called after. + +The use of the `#save` / `#save!` and the `#create` / `#create!` methods will trigger the execution of save callbacks. + +:::info +`before_save` and `after_save` are called for both new and existing records. `before_save` callbacks are always executed _before_ `before_create` or `before_update` callbacks. `after_save` callbacks on the other hand are always executed _after_ `after_create` or `after_update` callbacks. +::: + +### `before_delete` and `after_delete` + +`before_delete` callbacks are called before a record gets deleted while `after_delete` callbacks are called after. + +The use of the `#delete` method will trigger these callbacks. + +### `after_commit` + +`after_commit` callbacks are called after a record is created, updated, or deleted, but only after the corresponding SQL transaction has been committed to the database (which isn't the case for other `after_*` callbacks - See [Transactions](./transactions) for more details). For example: + +```crystal +after_commit :do_something +``` + +As mentioned previously, by default such callbacks will run in the context of record creations, updates, and deletions. That being said it is also possible to associate these callbacks with one or more specific actions only by using the `on` argument. For example: + +```crystal +after_commit :do_something, on: :create # Will run after creations only +after_commit :do_something, on: :update # Will run after updates only +after_commit :do_something, on: :update # Will run after saves (creations or updates) only +after_commit :do_something, on: :delete # Will run after deletions only +after_commit :do_something_else, on: [:create, :delete] # Will run after creations and deletions only +``` + +The actions supported by the `on` argument are `create`, `update`, `save`, and `delete`. + +### `after_rollback` + +`after_rollback` callbacks are called after a transaction is rolled back when a record is created, updated, or deleted. For example: + +```crystal +after_rollback :do_something +``` + +As mentioned previously, by default such callbacks will run in the context of record creations, updates, and deletions. That being said it is also possible to associate these callbacks with one or more specific actions only by using the `on` argument. For example: + +```crystal +after_rollback :do_something, on: :create # Will run after rolled back creations only +after_rollback :do_something, on: :update # Will run after rolled back updates only +after_rollback :do_something, on: :update # Will run after rolled back saves (creations or updates) only +after_rollback :do_something, on: :delete # Will run after rolled back deletions only +after_rollback :do_something_else, on: [:create, :delete] # Will run after rolled back creations and deletions only +``` + +The actions supported by the `on` argument are `create`, `update`, `save`, and `delete`. diff --git a/docs/versioned_docs/version-0.3/models-and-databases/how-to/create-custom-model-fields.md b/docs/versioned_docs/version-0.3/models-and-databases/how-to/create-custom-model-fields.md new file mode 100644 index 000000000..6210ebeaa --- /dev/null +++ b/docs/versioned_docs/version-0.3/models-and-databases/how-to/create-custom-model-fields.md @@ -0,0 +1,294 @@ +--- +title: Create custom model fields +description: How to create custom model fields. +--- + +Marten gives you the ability to create your own custom model field implementations, which can involve custom validation logics, errors, and custom behaviors. You can choose to leverage these custom fields as part of your project's model definitions, and you can even distribute them to let other projects use them. + +## Model fields: scope and responsibilities + +Model fields have the following responsibilities: + +* they define the type and properties of the underlying column at the database level +* they define the necessary Crystal bindings at the model class level: this means that they can contribute custom methods or instance variables to the models making use of them (eg. getters, setters, etc) +* they define how field values are validated and/or sanitized + +Creating a custom model field does not necessarily mean that all of these responsibilities need to be taken care of as part of the custom field implementation. It really depends on whether you want to: + +* leverage an existing built-in model field (eg. integer, string, etc) +* or create a new model field from scratch. + +## Registering new model fields + +Regardless of the approach you take in order to define new model field classes ([subclassing built-in fields](#subclassing-existing-model-fields), or [creating new ones from scratch](#creating-new-model-fields-from-scratch)), these classes must be registered to the Marten's global fields registry in order to make them available for use when defining models. + +To do so, you will have to call the [`Marten::DB::Field#register`](pathname:///api/0.3/Marten/DB/Field.html#register(id%2Cfield_klass)-macro) method with the identifier of the field you wish to use, and the actual field class. For example: + +```crystal +Marten::DB::Field.register(:foo, FooField) +``` + +The identifier you pass to `#register` can be a symbol or a string. This is the identifier that is then made available to model classes in order to define their fields: + +```crystal +class MyModel < Marten::DB::Model + field :id, :big_int, primary_key: true, auto: true + // highlight-next-line + field :test, :foo, blank: true, null: true +end +``` + +The call to `#register` can be made from anywhere in your codebase, but obviously, you will want to ensure that it is done before requiring your model classes: indeed, Marten will make the compilation of your project fail if it can't find the field type you are trying to use as part of a model definition. + +## Subclassing existing model fields + +The easiest way to introduce a model field is probably to subclass one of the [built-in model fields](../reference/fields) provided by Marten. This can make a lot of sense if the "type" of the field you are trying to implement is already supported by Marten. + +For example, implementing a custom "email" field could be done by subclassing the existing [`Marten::DB::Field::String`](pathname:///api/0.3/Marten/DB/Field/String.html) class. Indeed, an "email" field is essentially a string with a pre-defined maximum size and some additional validation logic: + +```crystal +class EmailField < Marten::DB::Field::String + def initialize( + @id : ::String, + @max_size : ::Int32 = 254, + @primary_key = false, + @default : ::String? = nil, + @blank = false, + @null = false, + @unique = false, + @index = false, + @db_column = nil + ) + end + + def validate(record, value) + return if !value.is_a?(::String) + + # Leverage string's built-in validations (max size). + super + + if !EmailValidator.valid?(value) + record.errors.add(id, "Provide a valid email address") + end + end + + macro check_definition(field_id, kwargs) + # No-op max_size automatic checks... + end +end +``` + +Everything that is described in the following section about [creating model fields from scratch](#creating-new-model-fields-from-scratch) also applies to the case of subclassing existing model fields: the same methods can be overridden if necessary, but leveraging an existing class can save you some work. + +## Creating new model fields from scratch + +Creating new model fields from scratch involves subclassing the [`Marten::DB::Field::Base`](pathname:///api/0.3/Marten/DB/Field/Base.html) abstract class. Because of this, the new field class is required to implement a set of mandatory methods. These mandatory methods, and some other ones that are optional (but interesting in terms of capabilities), are described in the following sections. + +### Mandatory methods + +#### `default` + +The `#default` method is responsible for returning the field's default value, if any. Not all fields support default values; if this does not apply to your field use case, you can simply "no-op" this method: + +```crystal +def default + # no-op +end +``` + +On the other hand, if your field can be initialized with a `default` argument (and if it defines a `@default` instance variable), another possibility is to define a `#default` getter: + +```crystal +getter default +``` + +#### `from_db` + +The `#from_db` method is responsible for converting the passed raw DB value to the right field value. Indeed, the value that is read from the database will usually need to be converted to another format. For example, a `uuid` field might need to convert a `String` value to a proper `UUID` object: + +```crystal +def from_db(value) : ::UUID? + case value + when Nil + value.as?(Nil) + when ::String + ::UUID.new(value.as(::String)) + when ::UUID + value.as(::UUID) + else + raise_unexpected_field_value(value) + end +end +``` + +It should be noted that you will usually want to handle the case of `nil` values as part of this method since fields can be configured as nullable via the [`null: true`](../reference/fields#null) option. + +If the value can't be processed properly by your field class, then it may be necessary to raise an exception. To do that you can leverage the `#raise_unexpected_field_value` method, which will raise a `Marten::DB::Errors::UnexpectedFieldValue` exception. + +#### `from_db_result_set` + +The `#from_db_result_set` method is responsible for extracting the field value from a DB result set and returning the right object corresponding to this value. This method will usually be called when retrieving your field's value from the database (when using the Marten ORM). The method takes a standard `DB::ResultSet` object as argument and it is expected that you use `#read` to retrieve the intended column value. See the [Crystal reference documentation](https://crystal-lang.org/reference/1.5/database/index.html#reading-query-results) for more details around these objects and methods. + +For example: + +```crystal +def from_db_result_set(result_set : ::DB::ResultSet) : ::UUID? + from_db(result_set.read(Nil | ::String | ::UUID)) +end +``` + +The `#from_db_result_set` method is supposed to return the read value into the right "representation", that is the final object representing the field value that users will interact with when manipulating model records (for example a `UUID` object created from a string). As such, you will usually want to call [`#from_db`](#fromdb) once you get the value from the database result set in order to return the final value. + +#### `to_column` + +Most model fields will contribute a corresponding column at the database level; these columns are read by Marten in order to generate migrations from model definitions. The column returned by the `#to_column` method should be an instance of a subclass of [`Marten::DB::Management::Column::Base`](pathname:///api/0.3/Marten/DB/Management/Column/Base.html). + +For example, an "email" field could return a string column as part of its `#to_column` method: + +```crystal +def to_column : Marten::DB::Management::Column::Base? + Marten::DB::Management::Column::String.new( + name: db_column!, + max_size: max_size, + primary_key: primary_key?, + null: null?, + unique: unique?, + index: index?, + default: to_db(default) + ) +end +``` + +If for some reason your custom field does not contribute any columns to the database model, it is possible to simply "no-op" the `#to_column` method by returning `nil` instead. + +#### `to_db` + +The `#to_db` method converts a field value from the "Crystal" representation to the database representation. As such, this method performs the reverse operation of the [`#from_db`](#fromdb) method. + +For example, this method could return the string representation of a `UUID` object: + +```crystal +def to_db(value) : ::DB::Any + case value + when Nil + nil + when ::UUID + value.hexstring + else + raise_unexpected_field_value(value) + end +end +``` + +Again, if the value can't be processed properly by the field class, it may be necessary to raise an exception. To do that you can leverage the `#raise_unexpected_field_value` method, which will raise a `Marten::DB::Errors::UnexpectedFieldValue` exception. + +### Other useful methods + +#### `initialize` + +The default `#initialize` method that is provided by the [`Marten::DB::Field::Base`](pathname:///api/0.3/Marten/DB/Field/Base.html) is fairly simply and looks like this: + +```crystal +def initialize( + @id : ::String, + @primary_key = false, + @blank = false, + @null = false, + @unique = false, + @index = false, + @db_column = nil +) +end +``` + +Depending on your field requirements, you might want to override this method completely in order to support additional parameters (such as default values, max sizes, validation-related options, etc). + +#### `validate` + +The `#validate` method does nothing by default and can be overridden on a per-field class basis in order to implement custom validation logic. This method takes the model record being validated and the field value as arguments, which allows you to easily run validation checks and to add [validation errors](../validations) to the model record. + +For example: + +```crystal +def validate(record, value) + return if !value.is_a?(::String) + + if !EmailValidator.valid?(value) + record.errors.add(id, "Provide a valid email address") + end +end +``` + +### An example + +Let's consider the use case of the "email" field highlighted in [Subclassing existing model fields](#subclassing-existing-model-fields). The exact same field could be implemented from scratch with the following snippet: + +```crystal +class EmailField < Marten::DB::Field::Base + getter default + getter max_size + + def initialize( + @id : ::String, + @max_size : ::Int32 = 254, + @primary_key = false, + @default : ::String? = nil, + @blank = false, + @null = false, + @unique = false, + @index = false, + @db_column = nil + ) + end + + def from_db(value) : ::String? + case value + when Nil | ::String + value.as?(Nil | ::String) + else + raise_unexpected_field_value(value) + end + end + + def from_db_result_set(result_set : ::DB::ResultSet) : ::String? + result_set.read(::String?) + end + + def to_column : Marten::DB::Management::Column::Base? + Marten::DB::Management::Column::String.new( + name: db_column!, + max_size: max_size, + primary_key: primary_key?, + null: null?, + unique: unique?, + index: index?, + default: to_db(default) + ) + end + + def to_db(value) : ::DB::Any + case value + when Nil + nil + when ::String + value + when Symbol + value.to_s + else + raise_unexpected_field_value(value) + end + end + + def validate(record, value) + return if !value.is_a?(::String) + + if value.size > @max_size + record.errors.add(id, "The maximum allowed length is #{@max_size} characters") + end + + if !EmailValidator.valid?(value) + record.errors.add(id, "Provide a valid email address") + end + end +end +``` diff --git a/docs/versioned_docs/version-0.3/models-and-databases/introduction.md b/docs/versioned_docs/version-0.3/models-and-databases/introduction.md new file mode 100644 index 000000000..d8a212837 --- /dev/null +++ b/docs/versioned_docs/version-0.3/models-and-databases/introduction.md @@ -0,0 +1,432 @@ +--- +title: Introduction to models +description: Learn how to define models and interact with model records. +sidebar_label: Introduction +--- + +Models define what data can be persisted and manipulated by a Marten application. They explicitly specify fields and rules that map to database tables and columns. As such, they correspond to the layer of the framework that is responsible for representing business data and logic. + +## Basic model definition + +Marten models must be defined as subclasses of the [`Marten::Model`](pathname:///api/0.3/Marten/DB/Model.html) base class; they explicitly define "fields" through the use of the `field` macro. These classes and fields map to database tables and columns that can be queried through the use of an automatically-generated database access API (see [Queries](./queries) for more details). + +For example, the following code snippet defines a simple `Article` model: + +```crystal +class Article < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :title, :string, max_size: 255 + field :content, :text +end +``` + +In the above example, `id`, `title`, and `content` are fields of the `Article` model. Each of these fields map to a database column in a table whose name is automatically inferred from the model name (and its associated application). If it was to be manually created using plain SQL, the `Article` model would correspond to the following statement (using the PostgreSQL syntax): + +```sql +CREATE TABLE myapp_articles ( + "id" bigserial NOT NULL PRIMARY KEY, + "title" varchar(255) NOT NULL, + "content" text NOT NULL +); +``` + +## Models and installed apps + +A model's application needs to be explicitly added to the list of installed applications for the considered project. Indeed, Marten requires projects to explicitly declare the applications they are using in the `installed_apps` configuration option. Model tables and migrations will only be created/applied for model classes that are provided by _installed apps_. + +For example, if the above `Article` model was associated with a `MyApp` application class, it would be possible to ensure that it is used by ensuring that the `installed_app` configuration option is as follows: + +```crystal +Marten.configure do |config| + config.installed_apps = [ + MyApp, + # other apps... + ] +end +``` + +## Model fields + +Model classes must define _fields_. Fields allow to specify the attributes of a model and they map to actual database columns. They are defined through the use of the `field` macro. + +For example: + +```crystal +class Author < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :first_name, :string, max_size: 255 + field :last_name, :string, max_size: 255 +end + +class Article < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :title, :string, max_size: 255 + field :content, :text + field :author, :many_to_one, to: Author +end +``` + +### Field ID and field type + +Every field in a model class must contain two mandatory positional arguments: a field identifier and a field type. + +The field identifier is used by Marten in order to determine the name of the corresponding database column. This identifier is also used to generate the Crystal bindings that allow you to interact with field values through getters and setters. + +The field type determines a few other things: + +* the type of the corresponding database column (for example `INTEGER`, `TEXT`, etc) +* the getter and setter methods that are generated for the field in the model class +* how field values are actually validated + +Marten provides numerous built-in field types that cover common web development needs. The complete list of supported fields is covered in the [model fields reference](./reference/fields). + +:::note +It is possible to write custom model fields and use them in your model definitions. See [How to create custom model fields](./how-to/create-custom-model-fields) for more details. +::: + +### Common field options + +In addition to their identifiers and types, fields can take keyword arguments that allow to further configure their behaviors and how they map to database columns. Most of the time those additional keyword arguments are optional, but they can be mandatory depending on the considered field type. + +Some of these optional field arguments are shared across all the available fields. Below is a list of the ones you'll encounter most frequently. + +#### `null` + +The `null` argument allows defining whether a field is allowed to store `NULL` values in the database. The default value for this argument is `false`. + +#### `blank` + +The `blank` argument allows defining whether a field is allowed to receive blank values from a validation perspective. The fields with `blank: false` that receive blank values will make their associated model record validation fail. The default value for this argument is `false`. + +#### `default` + +The `default` argument allows defining a default value for a given field. The default value for this argument is `nil`. + +#### `unique` + +The `unique` argument allows defining that values for a specific field must be unique throughout the associated table. The default value for this argument is `false`. + +### Mandatory primary key + +All Marten models must define one (and only one) primary key field. This primary key field will usually be an `int` or a `big_int` field using the `primary_key: true` and `auto: true` arguments, like in the following example: + +```crystal +class MyModel < Marten::Model + field :id, :big_int, primary_key: true, auto: true +end +``` + +It should be noted that the primary key can correspond to any other field type. For example, your primary key could correspond to an `uuid` field: + +```crystal +class MyModel < Marten::Model + field :id, :uuid, primary_key: true + + after_initialize :initialize_id + + def initialize_id + @id ||= UUID.random + end +end +``` + +### Relationships + +Marten provides special fields allowing to define the three most common types of database relationships: many-to-many, many-to-one, and one-to-one. + +#### Many-to-many relationships + +Many-to-many relationships can be defined through the use of [`many_to_many`](./reference/fields#many_to_many) fields. This special field type requires the use of a special `to` argument in order to specify the model class to which the current model is related. + +For example, an `Article` model could have a many-to-many field towards a `Tag` model. In such case, an `Article` record could have many associated `Tag` records, and every `Tag` record could be associated to many `Article` records as well: + +```crystal +class Tag < Marten::Model + # ... +end + +class Article < Marten::Model + # ... + field :tags, :many_to_many, to: Tag +end +``` + +#### Many-to-one relationships + +Many-to-one relationships can be defined through the use of [`many_to_one`](./reference/fields#many_to_one) fields. This special field type requires the use of a special `to` argument in order to specify the model class to which the current model is related. + +For example, an `Article` model could have a many-to-one field towards an `Author` model. In such case, an `Article` record would only have one associated `Author` record, but every `Author` record could be associated to many `Article` records: + +```crystal +class Author < Marten::Model + # ... +end + +class Article < Marten::Model + # ... + field :author, :many_to_one, to: Author +end +``` + +:::tip +It is possible to define recursive relationships by leveraging the `self` keyword. For example, if you want to define a many-to-one relationship field that targets that same model, you can do so easily by using `self` as the value for the `to` argument: + +```crystal +class TreeNode < Marten::Model + # ... + field :parent, :many_to_one, to: self +end +``` +::: + +#### One-to-one relationships + +One-to-one relationships can be defined through the use of [`one_to_one`](./reference/fields#one_to_one) fields. This special field type requires the use of a special `to` argument in order to specify the model class to which the current model is related. + +For example, a `User` model could have a one-to-one field towards a `Profile` model. In such case, the `User` model could only have one associated `Profile` record, and the reverse would be true as well (a `Profile` record could only have one associated `User` record). In fact, a one-to-one field is really similar to a many-to-one field, but with an additional unicity constraint: + +```crystal +class Profile < Marten::Model + # ... +end + +class User < Marten::Model + # ... + field :profile, :one_to_one, to: Profile +end +``` + +### Timestamps + +Marten lets you easily add automatic `created_at` / `updated_at` [`date_time`](./reference/fields#date_time) fields to your models by leveraging the [`#with_timestamp_fields`](pathname:///api/0.3/Marten/DB/Model/Table.html#with_timestamp_fields-macro) macro: + +```crystal +class Article < Marten::Model + // highlight-next-line + with_timestamp_fields + + field :id, :big_int, primary_key: true, auto: true + field :title, :string, max_size: 255 +end +``` + +The `created_at` field is populated with the current time when new records are created while the `updated_at` field is refreshed with the current time whenever records are updated. + +Note that using [`#with_timestamp_fields`](pathname:///api/0.3/Marten/DB/Model/Table.html#with_timestamp_fields-macro) is technically equivalent as defining two `created_at` and `updated_at` [`date_time`](./reference/fields#date_time) fields as follows: + +```crystal +class Article < Marten::Model + // highlight-next-line + field :created_at, :date_time, auto_now_add: true + // highlight-next-line + field :updated_at, :date_time, auto_now: true + + field :id, :big_int, primary_key: true, auto: true + field :title, :string, max_size: 255 +end +``` + +## Multifields indexes and unique constraints + +Single model fields can be indexed or associated with a unique constraint _individually_ by leveraging the [`index`](./reference/fields#index) and [`unique`](./reference/fields#unique) field options. That being said, it is sometimes necessary to configure multifields indexes or unique constraints. + +### Multifields indexes + +Multifields indexes can be configured in a model by leveraging the [`#db_index`](pathname:///api/0.3/Marten/DB/Model/Table/ClassMethods.html#db_index(name%3AString|Symbol%2Cfield_names%3AArray(String)|Array(Symbol))%3ANil-instance-method) class method. This method requires an index name argument as well as an array of targeted field names. + +For example: + +```crystal +class Person < Marten::Model + field :id, :int, primary_key: true, auto: true + field :first_name, :string, max_size: 50 + field :last_name, :string, max_size: 50 + + db_index :person_full_name_index, field_names: [:first_name, :last_name] +end +``` + +### Multifields unique constraints + +Multifields unique constraints can be configured in a model by leveraging the [`#db_unique_constraint`](pathname:///api/0.3/Marten/DB/Model/Table/ClassMethods.html#db_unique_constraint(name%3AString|Symbol%2Cfield_names%3AArray(String)|Array(Symbol))%3ANil-instance-method) class method. This method requires an index name argument as well as an array of targeted field names. + +For example: + +```crystal +class Booking < Marten::Model + field :id, :int, primary_key: true, auto: true + field :room, :string, max_size: 50 + field :date, :date, max_size: 50 + + db_unique_constraint :booking_room_date_constraint, field_names: [:room, :date] +end +``` + +The above constraint ensures that each room can only be booked for each date. + +## CRUD operations + +CRUD stands for **C**reate, **R**ead, **U**pdate, and **D**elete. Marten provides a set of methods and tools allowing applications to read and manipulate data stored in model tables. + +### Create + +Model records can be created through the use of the `#new` and `#create` methods. The `#new` method will simply initialize a new model record that is not persisted in the database. The `#create` method will initialize the new model record using the specified attributes and persist it to the database. + +For example, it would be possible to create a new `Article` model record by specifying its `title` and `content` attribute values through the use of the `#create` method as follows: + +```crystal +Article.create(title: "My article", content: "Learn how to build web apps with Marten!") +``` + +The same `Article` record could be initialized (but not saved!) through the use of the `new` method as follows: + +```crystal +Article.new(title: "My article", content: "Learn how to build web apps with Marten!") +``` + +It should be noted that field values can be assigned after a model instance has been initialized. For example, the previous example is equivalent to the following snippet: + +```crystal +article = Article.new +article.title = "My article" +article.content = "Learn how to build web apps with Marten!" +``` + +A model instance that was initialized like in the previous example will not be persisted to the database automatically. In this situation, it is possible to ensure that the corresponding record is created in the database by using the `#save` method (`article.save`). + +Finally it should be noted that both `#create` and `#new` support an optional block that will receive the initialized model record. This allows to initialize attributes or to call additional methods on the record being initialized: + +```crystal +Article.create do |article| + article.title = "My article" + article.content = "Learn how to build web apps with Marten!" +end +``` + +### Read + +Marten models provide a powerful API allowing to read and query records. This is achieved by constructing "query sets". A query set is a representation of records collections from the database that can be filtered. + +For example, it is possible to return a collection of all the `Article` model records using: + +```crystal +Article.all +``` + +It is possible to retrieve a specific record matching a set of filters (for example the value of an identifier) by using: + +```crystal +Article.get(id: 42) +``` + +Finally the following snippet showcases how to filter `Article` records by title and to sort them by creation date in reverse chronological order: + +```crystal +Article.filter(name: "My article").order("-created_at") +``` + +Please head over to the [Model queries](./queries) guide in order to learn more about model querying capabilities. + +### Update + +Once a model record has been retrieved from the database, it is possible to update it by modifying its attributes and calling the `#save` method: + +```crystal +article = Article.get(id: 42) +article.title = "Updated!" +article.save +``` + +Marten also provide the ability to update the records that are targetted by a specific query set through the use of the `#update` method, like in the following example: + +```crystal +Article.filter(title: "My article").update(title: "Updated!") +``` + +### Delete + +Once a model record has been retrieved from the database, it is possible to delete it by using the `#delete` method: + +```crystal +article = Article.get(id: 42) +article.delete +``` + +Marten also provide the ability to delete the records that are targetted by a specific query set through the use of the `#delete` method, like in the following example: + +```crystal +Article.filter(title: "My article").delete +``` + +## Validations + +Marten lets you specify how to validate model records before they are persisted to the database. These validation rules can be inherited from the fields in your model depending on the options you used (for example fields using `blank: false` will make the associated record validation fail if the field value is blank). They can also be explicitly specified in your model class, which is useful if you need to implement custom validation logics. + +For example: + +```crystal +class User < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :name, :string, max_size: 255 + + validate :validate_name + + private def validate_name + errors.add(:name, "Name must not be less than 3 characters!") if name && name!.size < 3 + end +end +``` + +Most of the methods presented above that actually persist model records to the database (like `#create` or `#save`) run these validation rules. This means that they will automatically validate the considered records before propagating any changes to the database. It should be noted that in the event that a record is invalid, these methods will return `false` to indicate that the considered object is invalid (and they will return `true` if the object is valid). The `#create` and `#save` methods also have bang counterparts (`#create!` and `#save!`) that will explicitly raise a validation error in case of invalid records: + +```crystal +article = Article.new +article.save +# => false +article.save! +# => Unhandled exception: Record is invalid (Marten::DB::Errors::InvalidRecord) +``` + +Please head over to the [Model validations](./validations) guide in order to learn more about model validations. + +## Inheritance + +Model classes can inherit from each other. This allows you to easily reuse the field definitions and table attributes of a parent model within a child model. + +Presently, the Marten web framework only allows [abstract model inheritance](#inheriting-from-abstract-models) (which is useful in order to reuse shared model fields and patterns over multiple child models without having a database table created for the parent model). Support for multi-table inheritance is planned for future releases. + +:::caution +You can technically inherit from concrete model classes, but this will result in the same behavior as the [abstract model technique](#inheriting-from-abstract-models). As mentioned previously, this behavior is likely to change in future Marten versions and you should probably not rely on it. +::: + +### Inheriting from abstract models + +You can define abstract model classes by leveraging [Crystal's abstract type mechanism](https://crystal-lang.org/reference/syntax_and_semantics/virtual_and_abstract_types.html). Doing so allows to easily reuse model field definitions, table properties, and custom logics within child models. In this situation, the parent's model does not contribute any table to the considered database. + +For example: + +```crystal +abstract class Person < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :name, :string, max_size: 255 + field :email, :string, max_size: 255 +end + +class Student < Person + field :grade, :string, max_size: 15 +end +``` + +The `Student` model will have four model fields in total (`id`, `name`, `email`, and `grade`). Moreover, all the methods of the parent model fields will be available on the child model. It should be noted that in this case the `Person` model cannot be used like a regular model: for example, trying to query records will return an error since no table is actually associated with the abstract model. Since it is an [abstract type](https://crystal-lang.org/reference/syntax_and_semantics/virtual_and_abstract_types.html), the `Student` class can't be instantiated either. + +## Callbacks + +It is possible to define callbacks in your model in order to bind methods and logics to specific events in the life cycle of your model records. For example, it is possible to define callbacks that run before a record gets created, or before it is destroyed. + +Please head over to the [Model callbacks](./callbacks) guide in order to learn more about model callbacks. + +## Migrations + +When working with models, it is necessary to ensure that any changes made to model definitions are applied at the database level. This is achieved through the use of migrations. + +Marten provides a migrations mechanism that is designed to be automatic: this means that migrations will be automatically generated from your model definitions when you run a dedicated command (the `genmigrations` command). Please head over to [Model migrations](./migrations) in order to learn more about migrations generations and the associated workflows. diff --git a/docs/versioned_docs/version-0.3/models-and-databases/migrations.md b/docs/versioned_docs/version-0.3/models-and-databases/migrations.md new file mode 100644 index 000000000..d1ed80e0d --- /dev/null +++ b/docs/versioned_docs/version-0.3/models-and-databases/migrations.md @@ -0,0 +1,358 @@ +--- +title: Model migrations +description: Learn how to generate and work with model migrations. +sidebar_label: Migrations +--- + + +Migrations ensure that any changes made to model definitions are applied at the database level. Marten's migrations mechanism is designed to be mostly automatic: migrations are generated from your model definitions when you run a dedicated command, after having introduced some changes to your model definitions. + +## Overview + +Marten is able to automatically create migrations when you introduce changes to your models. + +For example, let's assume you just added a `hometown` string field to an existing `Author` model: + +```crystal +class Author < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :first_name, :string, max_size: 255 + field :last_name, :string, max_size: 255 + // highlight-next-line + field :hometown, :string, max_size: 255, blank: true, null: true +end +``` + +After introducing this change, you can run the `genmigrations` management command as follows: + +```shell +$ marten genmigrations +Generating migrations for app 'blog': + › Creating [src/blog/migrations/202206221856241_add_hometown_to_blog_author_table.cr]... DONE + ○ Add hometown to blog_author table +``` + +When running this command, the table definition corresponding to your current model will be analyzed and compared to the equivalent table that is defined by your migration files. Depending on the result of this analysis, a new set of migration files will be generated in order to account for the changes that were made to the models. + +Generated migrations are saved under a `migrations` folder that lives in the local structure of every [application](../development/applications). As a general rule of thumb, you should always look at what is actually generated by the `genmigrations` command. Indeed, the introspection capabilities of this command could be limited depending on what you are trying to achieve. + +In the above example, the generated migration file looks something like this: + +```crystal +# Generated by Marten 0.1.0 on 2022-06-22 18:56:24 -04:00 + +class Migration::Blog::V202206221856241 < Marten::Migration + depends_on :blog, "202205290942181_previous_migration_name" + + def plan + add_column :blog_author, :hometown, :string, max_size: 255, null: true + end +end +``` + +As you see, the migration explicitly depends on the previous migration for the considered application (`blog` in this case) and it defines a migration "plan" that involves adding a `hometown` string column to the `blog_author` table. + +We can then apply this migration to the database by running the `migrate` Marten command: + +```shell +$ marten migrate +Running migrations: + › Applying blog_202206221856241_add_hometown_to_blog_author_table... DONE +``` + +## Available commands + +Marten provides a set of four commands allowing interacting with model migrations: + +* [`genmigrations`](../development/reference/management-commands#genmigrations) allows generating migration files by looking for changes in your model definitions +* [`migrate`](../development/reference/management-commands#migrate) allows applying (or unapplying) migrations to your databases +* [`listmigrations`](../development/reference/management-commands#listmigrations) allows listing all the migrations for all your [installed applications](../development/applications), and whether they have already been applied or not +* [`resetmigrations`](../development/reference/management-commands#resetmigrations) allows resetting multiple migrations into a single one + +## DB connections specificities + +Migrations can be used with all the built-in database backends provided by Marten: PostgreSQL, MySQL, and SQlite. That being said, there are some key differences among those backends that you should be aware of when it comes to migrations. These differences stem from the fact that not all of these databases support schema alteration operations or DDL (Data Definition Language) transactions. + +PostgreSQL and SQLite support DDL transactions, but MySQL does *not* support them. As such, if a migration fails when being applied to a MySQL database, you might have to manually undo some of the operations in order to try again. + +Finally, it should be noted that SQLite does not support most schema alteration operations. Because of this, Marten will perform the following set of operations when applying model changes to a SQLite database: + +1. create a new table corresponding to the new model definition +2. copy the data from the old table to the new one +3. delete the old table +4. rename the new one so that it matches the model's table name + +## Migration files + +As presented in the [overview](#overview) section above, migration files are automatically generated by Marten by identifying changes in your model definitions (although migrations could be created and defined manually if needed). These files are persisted in a `migrations` folder inside each application's main directory. + +Migrations always inherit from the [`Marten::Migration`](pathname:///api/0.3/Marten/DB/Migration.html) base class. A basic migration will look something like this: + +```crystal +# Generated by Marten 0.1.0 on 2022-03-13 16:08:37 -04:00 + +class Migration::Main::V202203131608371 < Marten::Migration + depends_on :users, "202203131607261_create_users_user_table" + depends_on :main, "202203131604261_add_content_to_main_article_table" + + def plan + add_column :main_article, :author_id, :reference, to_table: :users_user, to_column: :id, null: true + end +end +``` + +Each migration must define the following mandatory information: the migration's **dependencies** and the migration's **operations**. + +### Dependencies + +Marten migrations can depend upon one another. As such, each migration generally defines one or many dependencies through the use of the [`#depends_on`](pathname:///api/0.3/Marten/DB/Migration.html#depends_on(app_name%3AString|Symbol%2Cmigration_name%3AString|Symbol)-class-method) class method, which takes an application label and a migration name as positional arguments. + +Migration dependencies are used to ensure that changes to a database are applied in the right order. For example, the previous migration is part of the `main` app and depends on two other migrations: first, it depends on the previous migration of the `main` app (`202203131604261_add_content_to_main_article_table`). Secondly, it depends on the `202203131607261_create_users_user_table` migration of the `users` app. This makes sense considering that the migration's only operation is adding an `author_id` foreign key targetting the `users_user` table to the `main_article` table: the dependency instructs Marten to first apply the `202203131607261_create_users_user_table` migration (which creates the `users_user` table) before applying the migration adding the new foreign key (since this foreign key requires the targetted table to exist first in order to be created properly). + +Again, migration dependencies are automatically identified and generated by Marten when you create migrations through the use of the [`genmigrations`](../development/reference/management-commands#genmigrations) command. These dependencies are identified by looking at the DB relationships between the application for which migrations are generated for and the other installed applications. + +### Operations + +Migrations must define operations to apply (or unapply) at the database level. Unless instructed otherwise, these operations are all executed within a single transaction for the backends that support DDL transactions (PostgreSQL and SQLite). + +When they are generated automatically, migrations will define a set of operations to execute as part of a `#plan` method. This method can define one or more operations. For example: + +```crystal +# Generated by Marten 0.1.0 on 2022-03-30 22:13:06 -04:00 + +class Migration::Press::V202203302213061 < Marten::Migration + depends_on :press, "202203111822091_initial" + + def plan + add_column :press_article, :rating, :int, null: true + remove_column :press_article, :old_status + end +end +``` + +Migrations can be *applied* and *unapplied*. This means that when a migration is applied, all the operations defined in the `#plan` method will be executed in the order they were defined. But if the migration is unapplied, the exact same operation will be "reversed" in reverse order. + +In the previous example, the "plan" of the migration involves two operations: first, we are adding a new `rating` column to the `press_article` table and then we are removing an `old_status` column from the same table. If we were applying this migration, these two operations would be executed in this order. But if we were unapplying the migration, this means that we would first re-create the `old_status` column in the `press_article` table, and then we would remove the `rating` column from the `press_article` table. + +The bidirectional aspect of the `#plan` method can be leveraged for most migration use cases, but there could be situations where the operations involved when *applying* the migration differ from the operations involved when *unapplying* the migration. In such situations, forward operations can be defined in a `#forward` method while backward operations can be defined in a `#backward` method: + +```crystal +# Generated by Marten 0.1.0 on 2022-03-30 22:13:06 -04:00 + +class Migration::Press::V202203302213061 < Marten::Migration + depends_on :press, "202203111822091_initial" + + def forward + add_column :press_article, :rating, :int, null: true + remove_column :press_article, :old_status + end + + def backward + # do something else + end +end +``` + +## Generating migrations + +Generating migrations is possible through the use of the [`genmigrations`](../development/reference/management-commands#genmigrations) management command. This command will scan the table definition corresponding to your current models and will compare it to the equivalent table that is defined by your migration files. Based on the result of this analysis, a new set of migrations will be created and persisted in your applications' `migrations` folders. + +:::info +The `genmigrations` command can only create migrations for [installed applications](../development/applications). If the command is not detecting the intended changes, make sure that your models are part of an installed application. +::: + +Running `marten genmigrations` will generate migrations for *all* of the models provided by your installed applications, but it is possible to restrict the generation to a specific application label by specifying an additional argument as follows: + +```shell +$ marten genmigrations my_app +``` + +Another usefull command option is the `--empty` one, which allows generating an empty migration file (without any defined operations): + +```shell +$ marten genmigrations my_app --empty +``` + +## Applying and unapplying migrations + +As mentioned previously, migrations can be applied by running the [`migrate`](../development/reference/management-commands#migrate) management command. + +Running `marten migrate` will execute non-applied migrations for your installed applications. That being said, it is possible to ensure that only the migrations of a specific application are applied by specifying an additional argument as follows: + +```shell +$ marten migrate my_app +``` + +To unapply certain migrations (or to apply some of them up to a certain version only), it is possible to specify another argument corresponding to the version of a targetted migration. For example, we could unapply all the migrations after the `202203111821451` version for the `my_app` application by running: + +```shell +$ marten migrate my_app 202203111821451 +``` + +If you wish to unapply all the migrations of a specific application, you can do so by targeting the `zero` version: + +```shell +$ marten migrate my_app zero +``` + +Finally, it should be noted that it is possible to "fake" the fact that migrations are applied or unapplied by using the `--fake` command option. When doing so, only the fact that the migration was applied (or unapplied) will be registered, and no migration operations will be executed: + +```shell +$ marten migrate my_app 202203111821451 --fake +``` + +## Transactions + +The operations of a migrations will be executed inside a single transaction by default unless this capability is not supported by the database backend (which is the case for MySQL). + +It is possible to disable this default behavior by using the [`#atomic`](pathname:///api/0.3/Marten/DB/Migration.html#atomic(atomic%3ABool)-class-method) method, in the migration class: + +```crystal +# Generated by Marten 0.1.0 on 2022-03-30 22:13:06 -04:00 + +class Migration::Press::V202203302213061 < Marten::Migration + depends_on :press, "202203111822091_initial" + + atomic false + + def plan + add_column :press_article, :rating, :int, null: true + remove_column :press_article, :old_status + end +end +``` + +## Data migrations + +Sometimes, it is necessary to write migrations that don't change the database schema but that actually write data to the database. This is often the case when backfilling column values for example. In order to do so, Marten provides the ability to run arbitrary code as part of migrations through the use of a special `#run_code` operation. + +For example, the following migration will run the `#run_forward_code` method when applying the migration, and it will run the `#run_backward_code` method when unapplying it: + +```crystal +# Generated by Marten 0.1.0 on 2022-03-30 22:13:06 -04:00 + +class Migration::Press::V202203302213061 < Marten::Migration + depends_on :press, "202203111822091_initial" + + def plan + run_code :run_forward_code, :run_backward_code + end + + def run_forward_code + # do something + end + + def run_backward_code + # do something + end +end +``` + +`#run_code` can be called with a single argument if you don't want to specify a "backward" method. The following migration is technically equivalent to the one in the previous example: + +```crystal +# Generated by Marten 0.1.0 on 2022-03-30 22:13:06 -04:00 + +class Migration::Press::V202203302213061 < Marten::Migration + depends_on :press, "202203111822091_initial" + + def forward + run_code :run_forward_code + end + + def backward + run_code :run_backward_code + end + + def run_forward_code + # do something + end + + def run_backward_code + # do something + end +end +``` + +:::info +Data migration logics can't be defined as part of the `#forward` and `#backward` methods directly. Indeed, the `#plan`, `#forward`, and `#backard` methods don't actually do anything to the database: they simply build a plan of operations that Marten will use when applying or unapplying the migrations. That's why it is necessary to use the `#run_code` operation when defining data migrations. +::: + +It should be noted that one convenient way to start writing a data migration is to generate an empty migration skeleton for the targetted application: + +```shell +$ marten genmigrations my_app --empty +``` + +## Executing custom SQL statements + +Sometimes the built-in migration operations that are provided by Marten might not be enough and you may want to run arbitrary SQL statements on the database as part of migrations. + +To do so, you can leverage the [`#execute`](./reference/migration-operations#execute) migration operation. This operation must be called with a forward statement string as first positional argument, and it can also take a second positional (and optional) argument in order to specify the statement to execute when unapplying the migration (which is only useful when defining a bidirectional operation inside the `#plan` method). + +For example: + +```crystal +# Generated by Marten 0.1.0 on 2022-03-30 22:13:06 -04:00 + +class Migration::Press::V202203302213061 < Marten::Migration + depends_on :press, "202203111822091_initial" + + def plan + execute("CREATE EXTENSION 'uuid-ossp';") + end +end +``` + +## Resetting migrations + +When your application starts having a certain amount of migrations, it can become interesting to reset them. This means reducing all the existing migrations to a unique migration file. This can be achieved through the use of the [`resetmigrations`](../development/reference/management-commands#resetmigrations) management command. + +For example, let's consider the following situation where we have three migrations under the `my_app` application: + +```shell +$ marten listmigrations +[my_app] + [✔] my_app_202203111821451_create_my_app_article_table + [✔] my_app_202203111822091_add_status_to_my_app_article_table + [✔] my_app_202204051755101_add_rating_to_my_app_article_table +``` + +Running the `resetmigrations` command will produce the following output: + +```shell +$ marten resetmigrations my_app +Generating migrations for app 'my_app': + › Creating [src/my_app/migrations/202206261646061_auto.cr]... DONE + ○ Create my_app_article table +``` + +If we look at the generated migration, you will notice that it includes multiple calls to the [`#replaces`](pathname:///api/0.3/Marten/DB/Migration.html#replaces(app_name%3AString|Symbol%2Cmigration_name%3AString|Symbol)-class-method) class method: + +```crystal +# Generated by Marten 0.1.0 on 2022-06-26 16:46:06 -04:00 + +class Migration::MyApp::V202206261646061 < Marten::Migration + replaces :my_app, "202203111821451_create_my_app_article_table" + replaces :my_app, "202203111822091_add_status_to_my_app_article_table" + replaces :my_app, "202204051755101_add_rating_to_my_app_article_table" + + def plan + create_table :my_app_article do + column :id, :big_int, primary_key: true, auto: true + column :title, :string, max_size: 155 + column :body, :text + column :status, :string, max_size: 64, null: true + column :rating, :int, null: true + end + end +end +``` + +These `#replaces` method calls indicate the previous migrations that the current migration is replacing. This new migration can be committed to your project's repository. Then it can be applied like any other migration (even if the underlying database was already up to date with the latest migrations). The following migrations will use it as a dependency and will disregard all the migrations that were replaced. + +Later on, you can decide to remove the old migrations (the ones that were replaced by the new migration). But obviously, you should only do so after a certain while in order to give a chance to developers to apply all the latest migrations. + +:::warning +The `resetmigrations` management command will not carry on `run_code` operations nor any other "manually" added operations. As a matter of fact, `resetmigrations` will simply look at your model definitions and try to recreate a new migration file from the beginning (without considering your old migration files). As such, if your database requires special `run_code` or `run_sql` operations, you should make sure that those are added as following migrations. +::: diff --git a/docs/versioned_docs/version-0.3/models-and-databases/multiple-databases.md b/docs/versioned_docs/version-0.3/models-and-databases/multiple-databases.md new file mode 100644 index 000000000..754ee2225 --- /dev/null +++ b/docs/versioned_docs/version-0.3/models-and-databases/multiple-databases.md @@ -0,0 +1,67 @@ +--- +title: Multiple databases +description: Learn how to leverage multiple databases in a Marten project. +--- + +This section covers how to leverage multiple databases within a Marten project: how to configure these additional databases and how to query them. + +:::caution +Support for multi-database projects is still experimental and lacking features such as DB routing. +::: + +## Defining multiple databases + +Each Marten project leveraging a single database uses what is called a "default" database. This is the database whose configuration is defined when calling the [`#database`](pathname:///api/0.3/Marten/Conf/GlobalSettings.html#database(id%3DDB%3A%3AConnection%3A%3ADEFAULT_CONNECTION_NAME%2C%26)-instance-method) configuration method: + +```crystal +config.database do |db| + db.backend = :sqlite + db.name = "default_db.db" +end +``` + +The "default" database is implied whenever you interact with the database (eg. by performing queries, creating records, etc), unless specified otherwise. + +The [`#database`](pathname:///api/0.3/Marten/Conf/GlobalSettings.html#database(id%3DDB%3A%3AConnection%3A%3ADEFAULT_CONNECTION_NAME%2C%26)-instance-method) configuration method can take an additional argument in order to define additional databases. For example: + +```crystal +config.database :other_db do |db| + db.backend = :sqlite + db.name = "other_db.db" +end +``` + +Think of this additional argument as a "database identifier" or alias that you can choose and that will allow you to interact with this specific database later on. + +## Applying migrations to your databases + +The [`migrate`](../development/reference/management-commands#migrate) management command operates on the "default" database by default, but it also accepts an optional `--db` option that lets you specify to which database the migrations should be applied. The value you specify for this option must correspond to the alias you configured when defining your databases in your project's configuration. For example: + +```bash +marten migrate --db=other_db +``` + +Note that running such a command would apply **all** the migrations to the `other_db` database. There is presently no way to ensure that only specific models or migrations are applied to a particular database only. + +## Manually selecting databases + +Marten lets you select which database you want to use when performing model-related operations. Unless specified, the "default" database is always implied but it is possible to explicitly define to which database operations should be applied. + +### Querying records + +When querying records, you can use the [`#using`](./reference/query-set#using) query set method in order to specify the target database. For example: + +```crystal +Article.all # Will target the "default" database +Article.using(:other_db).all # Will target the "other_db" database +``` + +### Persisting records + +When creating, updating, or deleting records, it is possible to specify to which database the operation should be applied to by using the `using` argument. For example: + +```crystal +tag = Tag.new(label: "crystal") +tag.save(using: :other_db) +tag.delete(using: :other_db) +``` diff --git a/docs/versioned_docs/version-0.3/models-and-databases/queries.md b/docs/versioned_docs/version-0.3/models-and-databases/queries.md new file mode 100644 index 000000000..5bee746e6 --- /dev/null +++ b/docs/versioned_docs/version-0.3/models-and-databases/queries.md @@ -0,0 +1,368 @@ +--- +title: Querying model records +description: Learn how to query model records. +sidebar_label: Queries +--- + +Once [models are properly defined](./introduction), it is possible to leverage the querying API in order to interact with model records. This API lets you build what is commonly referred to as "query sets": that is, representations of records collections that can be read, filtered, updated, or deleted. + +This documents covers the main features of the [query set API](./reference/query-set). Most of the examples used to illustrate these features will refer to the following models: + +```crystal +class City < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :name, :string, max_size: 255 +end + +class Author < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :first_name, :string, max_size: 255 + field :last_name, :string, max_size: 255 + field :hometown, :foreign_key, to: City, blank: true, null: true +end + +class Article < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :title, :string, max_size: 255 + field :content, :text + field :author, :many_to_one, to: Author +end +``` + +## Creating new records + +New model records can be created through the use of the `#new` and `#create` methods. The `#new` method will simply initialize a new model record that is not persisted in the database. The `#create` method will initialize the new model record using the specified attributes and then persist it to the database. + +For example, it is possible to create a new `Author` model record by specifying their `first_name` and `last_name` attribute values through the use of the `create` method like this: + +```crystal +Author.create(first_name: "John", last_name: "Doe") +``` + +The same `Author` record could be initialized (but not saved!) through the use of the `new` method as follows: + +```crystal +Author.new(first_name: "John", last_name: "Doe") +``` + +In the previous example the model instance will not be persisted to the database automatically. In order to explicitly save it to the database it is possible to use the `#save` method: + +```crystal +author = Author.new(first_name: "John", last_name: "Doe") # not persisted yet! +author.save # the author is now persisted to the database! +``` + +Finally, it should be noted that both `#create` and `#new` support an optional block that will receive the initialized model record. This allows to initialize attributes or to call additional methods on the record being initialized: + +```crystal +Author.create do |author| + author.first_name = "John" + author.last_name = "Doe" +end +``` + +:::caution +Model records will be validated before being saved to the database. If this validation fails, both the `#create` and `#save` methods will silently fail: `#create` will return the invalid model instance while `#save` will return `false`. The `#create` and `#save` methods also have bang counterparts (`#create!` and `#save!`) that will explicitly raise a validation error (`Marten::DB::Errors::InvalidRecord`) in case of an invalid record. + +Please refer to [Validations](./validations) in order to learn more about model validations. +::: + +## Basic querying capabilities + +In order to interact with a collection of model records, it is necessary to construct a "query set". A query set represents a collection of model records in the database. It can have filters, be paginated, etc. Unless a specific "write" operation is performed on such query sets, they will usually be mapped to a standard `SELECT` statement where filters are converted to `WHERE` clauses. + +Query sets can be forged from a specific model by using methods such `#all`, `#filter`, or `#exclude` (those are described below). One of the key characteristics of query sets is that they are **lazily evaluated**: defining a query set will usually not involve any database operation. Additionally, most methods provided by query sets also return new query set objects. Query sets are only translated to SQL queries hitting the underlying database when records need to be extracted or manipulated by the considered codebase. + +For example, filters can be chained on a query set without it being evaluated. The query set is only evaluated when the actual records need to be displayed or when it becomes necessary to interact with them: + +```crystal +qset = Article.filter(title__startswith: "Top") # the query set is not evaluated +qset = qset.filter(author__first_name: "John") # the query set is not evaluated +puts qset # the query set is evaluated +``` + +In the above example, the two filters are simply chained without these resulting in database hits. The query set is only evaluated when the actual records need to be printed. + +Query sets are **iterable**: they provide the ability to iterate over the resulting records (which will also force the query set to be evaluated when this happens): + +```crystal +qset = Article.filter(title__startswith: "Top") # the query set is not evaluated +qset.each { |article| puts article } # the query set is evaluated +``` + +### Querying all records + +Retrieving all the records of a specific model can be achieved through the use of the `#all` method: + +```crystal +Author.all +``` + +"All records" does not necessarily mean all the records in the considered table. For example, `#all` can be chained to an existing query set that was filtered (which is usually unnecessary since this does not alter the resulting records): + +```crystal +Author.filter(first_name: "John").all +``` + +### Filtering specific records + +Filtering records is achieved through the use of the `#filter` method. The `#filter` method requires one or many predicate keyword arguments (in the format described in [Field predicates](#field-predicates)). For example: + +```crystal +Author.filter(first_name: "John") +``` + +The above query set will return `Author` records whose first name is "John". + +It’s possible to filter records using multiple filters. For example, the following queries are equivalent: + +```crystal +Author.filter(first_name: "John").filter(last_name: "Doe") +Author.filter(first_name: "John", last_name: "Doe") +``` + +By default, filters involving multiple parameters like in the above examples always produce SQL queries whose parameters are "AND"ed together. More complex queries (eg. using OR, NOT conditions) can be achieved through the use of the `q` DSL (which is described in [Complex filters with `q` expressions](#complex-filters-with-q-expressions)), as outlined by the following examples: + +```crystal +# Get Author records with either "Bob" or "Alice" as first name +Author.filter { q(first_name: "Bob") | q(first_name: "Alice") } + +# Get Author records whose first names are not "John" +Author.filter { -q(first_name: "Alice") } +``` + +### Excluding specific records + +Excluding records is achieved through the use of the `#exclude` method. This method provides exactly the same API as the [`#filter`](#filtering-specific-records) method outlined previously. It requires one or many predicate keyword arguments (in the format described in [Field predicates](#field-predicates)). For example: + +```crystal +Author.exclude(first_name: "John") + +Author.exclude(first_name: "John").exclude(last_name: "Doe") +Author.exclude(first_name: "John", last_name: "Doe") + +Author.exclude { q(first_name: "Bob") | q(first_name: "Alice") } +``` + +### Retrieving a specific record + +Retrieving a specific record is achieved through the use of the `#get` method, which requires one or many predicate keyword arguments (in the format described in [Field predicates](#field-predicates)). For example: + +```crystal +Author.get(id: 1) +Author.get(first_name: "John") +``` + +If the record is not found, `nil` will be returned. It should be noted that a bang version of this method also exists: `#get!`. This alternative method raises a `Marten::DB::Errors::RecordNotFound` error if the record is not found. Regardless of the method used, if multiple records are found for the passed predicates, a `Marten::DB::Errors::MultipleRecordsFound` error is raised. + +It is also possible to chain a `#get` call on a query set that was already filtered: + +```crystal +Author.filter(first_name: "John").get(id: 42) +``` + +### Retrieving the first or last record + +The `#first` and `#last` methods can be used to retrieve the first or last record for a given query set. + +```crystal +Author.filter(first_name: "John").first +Author.filter(first_name: "John").last +``` + +If the considered query set is empty, the returned value will be `nil`. It should be noted that these methods have a bang equivalent (`#first!` and `#last!`) that both raise a `NilAssertionError` if the query set is empty. + +### Field predicates + +Field predicates allow to define filters that are applied to a given query set. They map to `WHERE` clauses in the produced SQL queries. + +For example: + +```crystal +Article.filter(title__icontains: "top") +``` + +Will translate to a SQL query like the following one (using PostgreSQL's syntax): + +```sql +SELECT * FROM articles WHERE title LIKE UPPER("top") +``` + +Field predicates always contain a mandatory field name (`title` in the previous example) and an optional predicate type (`icontains` in the previous example). The field name and the predicate type are always separated by a double underscore notation (`__`). This notation (`__`) is used as the keyword argument name while the argument value is used to define the value to use to perform the filtering. + +:::tip +The field name can correspond to any of the fields defined in the model being filtered. For `many_to_one` or `one_to_one` fields, it's possible to append a `_id` at the end of the field name to explicitly filter on the raw ID of the related record: + +```crystal +Article.all.filter(author_id: 42) +``` +::: + +Marten support numerous predicate types, which are all documented in the [field predicates reference](./reference/query-set#field-predicates). The ones that you'll encounter most frequently are outlined below: + +#### `exact` + +The `exact` field predicate can be used for "exact" matches: only records whose field values exactly match the specified value will be returned. This is the default predicate type, and it's not necessary to specify it when filtering model records. + +As such, the two following examples are equivalent: + +```crystal +Author.filter(first_name: "John") +Author.filter(first_name__exact: "John") +``` + +#### `iexact` + +This field predicate can be used for case insensitive matches. + +For example, the following filter would return `Article` records whose titles are `Test`, `TEST` or `test`: + +```crystal +Article.filter(title__iexact: "test") +``` + +#### `contains` + +This field predicate can be used to filter strings that should contain a specific value. For example: + +```crystal +Article.filter(title__contains: "top") +``` + +A case insensitive equivalent (`icontains`) is also available. + +## Advanced querying capabilities + +### Complex filters with `q` expressions + +As mentioned previously, field predicates expressed as keyword arguments will use an AND operator in the produced `WHERE` clauses. In order to produce conditions using other operators, it is necessary to use `q` expressions. + +In order to produce such expressions, methods like `#filter`, `#exclude`, or `#get` can receive a block allowing to define complex conditions. Inside of this block, a `#q` method can be used in order to define conditions nodes that can be combined together using the following operators: + +* `&` in order to perform a logical "AND" +* `|` in order to perform a logical "OR" +* `-` in order to perform a logical negation + +For example, the following snippet will return all the `Article` records whose title starts with "Top" or "10": + +```crystal +Article.filter { q(title__startswith: "Top") | q(title__startswith: "10") } +``` + +Using this approach, it is possible to produce complex conditions by combining `q()` expressions with the `&`, `|`, and `-` operators. Parentheses can also be used to group statements: + +```crystal +Article.filter { + (q(title__startswith: "Top") | q(title__startswith: "10")) & -q(author__first_name: "John") +} +``` + +Finally it should be noted that you can define many field predicates _inside_ `q()` expressions. When doing so, the field predicates will be "AND"ed together: + +```crystal +Article.filter { + q(title__startswith: "Top") & -q(author__first_name: "John", author__last_name: "Doe") +} +``` + +### Joins and filtering relations + +The double underscores notation described previously (`__`) can also be used to filter based on related model fields. For example, in the considered models definitions, we have an `Article` model which defines a relation (`many_to_one` field) to the `Author` model through the `author` field. The `Author` model itself also defines a relation to a `City` record through the `hometown` field. + +Given this data model, we could easily retrieve `Article` records whose author's first name is "John" with the following query set: + +```crystal +Article.filter(author__first_name: "John") +``` + +We could even retrieve all the `Article` records whose author are located in "Montréal" with the following query set: + +```crystal +Article.filter(author__hometown__name: "Montreal") +``` + +And obviously, the above query sets could also be used along with more specific field predicate types. For example: + +```crystal +Author.filter(author__hometown__name__startswith: "New") +``` + +When doing “deep filtering” like this, related model tables are automatically "joined" at the SQL level (inner joins or left outer joins depending on the nullability of the filtered fields). So the filtered relations are also already "selected" as part of the query, and fully initialized at the Crystal level. + +It is also possible to explicitly define that a specific query set must "join" a set of relations. This can result in nice performance improvements since this can help reduce the number of SQL queries performed for a given codebase. This is achieved through the use of the `#join` method: + +```crystal +author_1 = Author.filter(first_name: "John") +puts author_1.hometown # DB hit to retrieve the associated City record + +author_2 = Author.join(:hometown).filter(first_name: "John") +puts author_2.hometown # No additional DB hit +``` + +The double underscores notations can also be used in the context of joins. For example: + +```crystal +# The associated Author and City records will be selected and fully initialized +# with the selected Article record. +Article.join(:author__hometown).get(id: 42) +``` + +### Pagination + +Marten provides a pagination mechanism that you can leverage in order to easily iterate over records that are split across several pages of data. This works as follows: each query set object lets you generate a "paginator" (instance of [`Marten::DB::Query::Paginator`](pathname:///api/0.3/Marten/DB/Query/Paginator.html)) from a given page size (the number of records you would like on each page). You can then use this paginator in order to request specific pages, which gives you access to the corresponding records and to some additional pagination metadata. + +For example: + +```crystal +query_set = Article.filter(published: true) + +paginator = query_set.paginator(10) +paginator.page_size # => 10 +paginator.pages_count # => 6 + +# Retrieve the first page and iterate over the underlying records +page = paginator.page(1) +page.each { |article| puts article } +page.number # 1 +page.previous_page? # => false +page.previous_page_number # => nil +page.next_page? # => true +page.next_page_number # => 2 +``` + +As you can see, paginator objects let you request specific pages by providing a page number (1-indexed!) to the [`#page`](pathname:///api/0.3/Marten/DB/Query/Paginator.html#page(number%3AInt)-instance-method) method. Such pages are instances of [`Marten::DB::Query::Page`](pathname:///api/0.3/Marten/DB/Query/Page.html) and give you the ability to easily iterate over the corresponding records. They also give you the ability to retrieve some pagination-related information (eg. about the previous and next pages by leveraging the [`#previous_page?`](pathname:///api/0.3/Marten/DB/Query/Page.html#previous_page%3F-instance-method), [`#previous_page_number`](pathname:///api/0.3/Marten/DB/Query/Page.html#previous_page_number-instance-method), [`#next_page?`](pathname:///api/0.3/Marten/DB/Query/Page.html#next_page%3F-instance-method), and [`#next_page_number`](pathname:///api/0.3/Marten/DB/Query/Page.html#next_page_number-instance-method) methods). + +## Updating records + +Once a model record has been retrieved from the database, it is possible to update it by modifying its attributes and calling the `#save` method (already mentioned previously): + +```crystal +article = Article.get(id: 42) +article.title = "Updated!" +article.save +``` + +It is also possible to update records through the use of query sets. To do so, the `#update` method can be chained to a pre-defined query set in order to update all the resulting records: + +```crystal +Article.filter(title: "My article").update(title: "Updated!") +``` + +When calling the `#update` method like in the previous example, the update is done at the SQL level (using a regular `UPDATE` SQL statement) and the method returns the number of impacted records. As such it's important to remember that records updated like this won't be instantiated or validated before the update, and that no callbacks will be executed for them. + +## Deleting records + +Single model records that have been retrieved from the database can be deleted by using the `#delete` method: + +```crystal +article = Article.get(id: 42) +article.delete +``` + +Marten also provide the ability to delete the records that are targetted by a specific query set through the use of the `#delete` method, like in the following example: + +```crystal +Article.filter(title: "My article").delete +``` + +By default, related objects that are associated with the deleted records will also be deleted by following the deletion strategy defined in each relation field (`on_delete` option, see the [reference](./reference/fields#on_delete) for more details). The method always returns the number of deleted records. diff --git a/docs/versioned_docs/version-0.3/models-and-databases/raw-sql.md b/docs/versioned_docs/version-0.3/models-and-databases/raw-sql.md new file mode 100644 index 000000000..91a3ae363 --- /dev/null +++ b/docs/versioned_docs/version-0.3/models-and-databases/raw-sql.md @@ -0,0 +1,99 @@ +--- +title: Performing raw SQL queries +description: Learn how to perform raw SQL queries. +sidebar_label: Raw SQL +--- + +Marten gives you the ability to execute raw SQL if the capabilities provided by [query sets](./queries) are not sufficient for the task at hand. When doing so, multiple solutions can be considered: you can either decide to perform raw queries that are mapped to actual model instances, or you can execute entirely custom SQL statements. + +## Performing raw queries + +It is possible to perform raw SQL queries and expect to have the corresponding records mapped to actual model instances. This is possible by using the [`#raw`](./reference/query-set#raw) query set method. + +For example, the following snippet would allow iterating over all the `Article` model records (by assuming that the corresponding database table is the `main_article` one): + +```crystal +Article.raw("select * from main_article").each do |article| + # Do something with `article` record +end +``` + +:::tip +You need to know the name of the model table you are targetting to use the [`#raw`](./reference/query-set#raw) query set method. Unless you have explicitly overridden this name by using the [`#db_table`](pathname:///api/0.3/Marten/DB/Model/Table/ClassMethods.html#db_table(db_table%3AString|Symbol)-instance-method) class method, the name of the model table is automatically generated by Marten using the following format: `_` (`model_name` being the underscore version of the model class name). +::: + +It should be noted that you can also "inject" parameters into your SQL query. To do so you have two options: either you specify these parameters as positional arguments, or you specify them as named arguments. Positional parameters must be specified using the `?` syntax while named parameters must be specified using the `:param` format. + +For example, the following query uses positional parameters: + +```crystal +Article.raw("SELECT * FROM articles WHERE title = ? and created_at > ?", "Hello World!", "2022-10-30") +``` + +And the following one uses named parameters: + +```crystal +Article.raw( + "SELECT * FROM articles WHERE title = :title and created_at > :created_at", + title: "Hello World!", + created_at: "2022-10-30" +) +``` + +:::caution +**Do not use string interpolations in your SQL queries!** + +You should never use string interpolations in your raw SQL queries as this would expose your code to SQL injection attacks (where attackers can inject and execute arbitrary SQL into your database). + +As such, never - ever - do something like that: + +```crystal +Article.raw("SELECT * FROM articles WHERE title = '#{title}'") +``` + +And instead, do something like that: + +```crystal +Article.raw("SELECT * FROM articles WHERE title = ?", title) +``` + +Also, note that the parameters are left **unquoted** in the raw SQL queries: this is very important as not doing it would expose your code to SQL injection vulnerabilities as well. Parameters are quoted automatically by the underlying database backend. +::: + +Finally, it should be noted that Marten does not validate the SQL queries you specify to the [`#raw`](./reference/query-set#raw) query set method. It is the developer's responsibility to ensure that these queries are (i) valid and (ii) that they return records that correspond to the considered model. + +## Executing other SQL statements + +If it is necessary to execute other SQL statements that don't fall into the scope of what's provided by the [`#raw`](./reference/query-set#raw) query set method, then it's possible to rely on the low-level DB connection capabilities. + +Marten's DB connections are essentially wrappers around DB connections provided by the [crystal-db](https://github.com/crystal-lang/crystal-db) package. They can be opened, which allows you to essentially execute any query on the considered database. + +For example, the following snippet would open a connection to the default database and execute a simple query: + +```crystal +Marten::DB::Connection.default.open do |db| + db.scalar("SELECT 1") +end +``` + +:::tip +If you are using multiple databases and need to execute SQL statements on a database that is not the default one, then you can retrieve the considered DB connection object by using the [`Marten::DB::Connection#get`](pathname:///api/0.3/Marten/DB/Connection.html#get(db_alias%3AString|Symbol)-class-method) method. This method simply requires an argument corresponding to the DB alias you want to retrieve (ie. the alias you assigned to the database in the [databases configuration](../development/reference/settings#database-settings)) and returns the corresponding DB connection: + +```crystal +db = Marten::DB::Connection.get(:other_db) + +db.open do |db| + db.scalar("SELECT 1") +end +``` +::: + +The [`#open`](pathname:///api/0.3/Marten/DB/Connection/Base.html#open(%26)-instance-method) method allows opening a connection to the considered database, which you can then use to perform queries. This method leverages Crystal's [DB opening mechanism](https://crystal-lang.org/reference/database/index.html#open-database) and it returns the same DB connection objects that you would get if you were using `DB#open` directly: + +```crystal +Marten::DB::Connection.default.open do |db| + db.exec "create table contacts (name varchar(30), age int)" +end +``` + +Please refer to [Crystal's official documentation on interacting with databases](https://crystal-lang.org/reference/database/index.html) to learn more about this low-level API. diff --git a/docs/versioned_docs/version-0.3/models-and-databases/reference/fields.md b/docs/versioned_docs/version-0.3/models-and-databases/reference/fields.md new file mode 100644 index 000000000..d40b57c6b --- /dev/null +++ b/docs/versioned_docs/version-0.3/models-and-databases/reference/fields.md @@ -0,0 +1,373 @@ +--- +title: Model fields +description: Model fields reference. +--- + +This page provides a reference for all the available field options and field types that can be used when defining models. + +## Common field options + +The following field options can be used for all the available field types when declaring model fields using the `field` macro. + +### `blank` + +The `blank` argument allows to define whether a field is allowed to receive blank values from a validation perspective. The fields with `blank: false` that receive blank values will make their associated model record validation fail. The default value for this argument is `false`. + +### `db_column` + +The `db_column` argument can be used to specify the name of the column corresponding to the field at the database level. Unless specified, the database column name will correspond to the field name. + +### `default` + +The `default` argument allows to define a default value for a given field. The default value for this argument is `nil`. + +### `index` + +The `index` argument can be used to specify that a database index must be created for the corresponding column. The default value for this argument is `false`. + +### `primary_key` + +The `primary_key` argument can be used to specify that a field corresponds to the primary key of the considered model table. The default value for this argument is `false`. + +### `null` + +The `null` argument allows to define whether a field is allowed to store `NULL` values in the database. The default value for this argument is `false`. + +### `unique` + +The `unique` argument allows to define that values for a specific field must be unique throughout the associated table. The default value for this argument is `false`. + +## Field types + +### `big_int` + +A `big_int` field allows persisting 64-bit integers. In addition to the [common field options](#common-field-options), such fields support the following arguments: + +#### `auto` + +The `auto` argument auto-increment for the considered database column. Defaults to `false`. + +This argument will be used mainly when defining integer IDs that automatically increment: + +```crystal +class MyModel < Marten::Model + field :id, :big_int, primary_key: true, auto: true + # ... +end +``` + +### `bool` + +A `bool` field allows persisting booleans. + +### `date` + +A `date` field allows persisting date values, which map to `Time` objects in Crystal. In addition to the [common field options](#common-field-options), such fields support the following arguments: + +#### `auto_now` + +The `auto_now` argument allows ensuring that the corresponding field value is automatically set to the current time every time a record is saved. This provides a convenient way to define `updated_at` fields. Defaults to `false`. + +#### `auto_now_add` + +The `auto_now_add` argument allows ensuring that the corresponding field value is automatically set to the current time every time a record is created. This provides a convenient way to define `created_at` fields. Defaults to `false`. + +### `date_time` + +A `date_time` field allows persisting date-time values, which map to `Time` objects in Crystal. In addition to the [common field options](#common-field-options), such fields support the following arguments: + +#### `auto_now` + +The `auto_now` argument allows ensuring that the corresponding field value is automatically set to the current time every time a record is saved. This provides a convenient way to define `updated_at` fields. Defaults to `false`. + +#### `auto_now_add` + +The `auto_now_add` argument allows ensuring that the corresponding field value is automatically set to the current time every time a record is created. This provides a convenient way to define `created_at` fields. Defaults to `false`. + +### `duration` + +A `duration` field allows persisting duration values, which map to [`Time::Span`](https://crystal-lang.org/api/Time/Span.html) objects in Crystal. `duration` fields are persisted as big integer values (number of nanoseconds) at the database level. + +### `email` + +An `email` field allows to persist _valid_ email addresses. In addition to the [common field options](#common-field-options), such fields support the following arguments: + +#### `max_size` + +The `max_size` argument is optional and defaults to 254 characters (in accordance with RFCs 3696 and 5321). It allows to specify the maximum size of the persisted email addresses. This maximum size is used for the corresponding column definition and when it comes to validate field values. + +### `file` + +A `file` field allows persisting the reference to an uploaded file. + +:::info +`file` fields can't be configured as primary keys. +::: + +#### `storage` + +This optional argument can be used to configure the storage that will be used to persist the actual files. It defaults to the media files storage (configured via the `media_files.storage` setting), but can be overridden on a per-field basis if needed: + +```crystal +my_storage = Marten::Core::Storage::FileSystem.new(root: "files", base_url: "/files/") + +class Attachment < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :file, :file, storage: my_storage +end +``` + +Please refer to [Managing files](../../files/managing-files) for more details on how to manage uploaded files and the associated storages. + +#### `upload_to` + +This optional argument can be used to configure where the uploaded files are persisted in the storage. It defaults to an empty string and can be set to either a string or a proc. + +If set to a string, it allows to define in which directory of the underlying storage files will be persisted: + +```crystal +class Attachment < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :file, :file, upload_to: "foo/bar" +end +``` + +If set to a proc, it allows to customize the logic allowing to generate the resulting path _and_ filename: + +```crystal +class Attachment < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :file, :file, upload_to: ->(filename : String) { File.join("files/uploads", filename) } +end +``` + +### `float` + +A `float` field allows persisting floating point numbers (`Float64` objects). + +### `int` + +An `int` field allows persisting 32-bit integers. In addition to the [common field options](#common-field-options), such fields support the following arguments: + +#### `auto` + +The `auto` argument auto-increment for the considered database column. Defaults to `false`. + +This argument will be used mainly when defining integer IDs that automatically increment: + +```crystal +class MyModel < Marten::Model + field :id, :int, primary_key: true, auto: true + # ... +end +``` + +### `json` + +A `json` field allows persisting JSON values to the database. + +JSON values are automatically parsed from the underlying database column and exposed as a [`JSON::Any`](https://crystal-lang.org/api/JSON/Any.html) object (or `nil` if no values are available) by default in Crystal: + +```crystal +class MyModel < Marten::Model + # Other fields... + field :metadata, :json +end + +MyModel.last!.metadata # => JSON::Any object +``` + +Additionally, it is also possible to specify a [`serializable`](#serializable) option in order to specify a class that makes use of [`JSON::Serializable`](https://crystal-lang.org/api/JSON/Serializable.html). When doing so, the parsing of the JSON values will result in the initialization of the corresponding serializable objects: + +```crystal +class MySerializable + include JSON::Serializable + + property a : Int32 | Nil + property b : String | Nil +end + +class MyModel < Marten::Model + # Other fields... + field :metadata, :json, serializable: MySerializable +end + +MyModel.last!.metadata # => MySerializable object +``` + +:::info +It should be noted that `json` fields are mapped to: + +* `jsonb` columns in PostgreSQL databases +* `text` columns in MySQL databases +* `text` columns in SQLite databases +::: + +#### `serializable` + +The `serializable` arguments allows to specify that a class making use of [`JSON::Serializable`](https://crystal-lang.org/api/JSON/Serializable.html) should be used in order to parse the JSON values for the model field at hand. When specifying a `serializable` class, the values returned for the considered model fields will be instances of that class instead of [`JSON::Any`](https://crystal-lang.org/api/JSON/Any.html) objects. + +### `string` + +A `string` field allows to persist small or medium string values. In addition to the [common field options](#common-field-options), such fields support the following arguments: + +#### `max_size` + +The `max_size` argument **is required** and allows to specify the maximum size of the persisted string. This maximum size is used for the corresponding column definition and when it comes to validate field values. + +#### `min_size` + +The `min_size` argument allows defining the minimum size allowed for the persisted string. The default value for this argument is `nil`, which means that the minimum size is not validated by default. + +### `text` + +A `text` field allows to persist large text values. In addition to the [common field options](#common-field-options), such fields support the following arguments: + +#### `max_size` + +The `max_size` argument allows to specify the maximum size of the persisted string. This maximum size is used when it comes to validate field values. Defaults to `nil`. + +### `uuid` + +A `uuid` field allows persisting Universally Unique IDentifiers (`UUID` objects). + +## Relationship field types + +### `many_to_many` + +A `many_to_many` field allows to define a many-to-many relationship. This special field type requires the use of a special `to` argument in order to specify the model class to which the current model is related. + +For example, an `Article` model could have a many-to-many field towards a `Tag` model. In such case, an `Article` record could have many associated `Tag` records, and every `Tag` records could be associated to many `Article` records as well: + +```crystal +class Tag < Marten::Model + # ... +end + +class Article < Marten::Model + # ... + field :tags, :many_to_many, to: Tag +end +``` + +In addition to the [common field options](#common-field-options), such fields support the following arguments: + +#### `to` + +The `to` argument **is required** and allows to specify the model class that is related to the model where the `many_to_many` field is defined. + +#### `related` + +The `related` argument allows defining the name of the reverse (or backward) relation on the targetted model. If we consider the previous example, it could be possible to define an `articles` backward relation in order to let `Tag` records expose their related `Article` records: + +```crystal +class Tag < Marten::Model + # ... +end + +class Article < Marten::Model + # ... + field :tags, :many_to_many, to: Tag, related: :articles +end +``` + +When the `related` argument is used, a method will be automatically created on the targetted model by using the chosen argument's value. For example, this means that all the `Article` records using a specific `Tag` record could be accessed through the use of the `Tag#articles` method in the previous snippet. + +The default value is `nil`, which means that no reverse relation is defined on the targetted model by default. + +### `many_to_one` + +A `many_to_one` field allows defining a many-to-one relationship. This special field type requires the use of a special `to` argument in order to specify the model class to which the current model is related. + +For example, an `Article` model could have a many-to-one field towards an `Author` model. In such case, an `Article` record would only have one associated `Author` record, but every `Author` record could be associated to many `Article` records: + +```crystal +class Author < Marten::Model + # ... +end + +class Article < Marten::Model + # ... + field :author, :many_to_one, to: Author +end +``` + +In addition to the [common field options](#common-field-options), such fields support the following arguments: + +#### `to` + +The `to` argument **is required** and allows to specify the model class that is related to the model where the `many_to_one` field is defined. + +#### `related` + +The `related` argument allows defining the name of the reverse (or backward) relation on the targetted model. If we consider the previous example, it could be possible to define an `articles` backward relation in order to let `Author` records expose their related `Article` records: + +```crystal +class Author < Marten::Model + # ... +end + +class Article < Marten::Model + # ... + field :author, :many_to_one, to: Author, related: :articles +end +``` + +When the `related` argument is used, a method will be automatically created on the targetted model by using the chosen argument's value. For example, this means that all the `Article` records associated with a specific `Author` record could be accessed through the use of the `Author#articles` method in the previous snippet. + +The default value is `nil`, which means that no reverse relation is defined on the targetted model by default. + +#### `on_delete` + +The `on_delete` argument allows to specify the deletion strategy to adopt when a related record (one that is targeted by the `many_to_one` field) is deleted. The following strategies can be specified (as symbols): + +* `:do_nothing`: is the default strategy. With this strategy, Marten won't do anything to ensure that records referencing the record being deleted are deleted or updated. If the database enforces referential integrity (which will be the case for foreign key fields), this means that deleting a record could result in database errors +* `:cascade`: this strategy can be used to perform cascade deletions. When deleting a record, Marten will try to first destroy the other records that reference the object being deleted +* `:protect`: this strategy allows to explicitly prevent the deletion of records if they are referenced by other records. This means that attempting to delete a "protected" record will result in a `Marten::DB::Errors::ProtectedRecord` error +* `:set_null`: this strategy will set the reference column to `null` when the related is deleted + +### `one_to_one` + +A `one_to_one` field allows defining a one-to-one relationship. This special field type requires the use of a special `to` argument in order to specify the model class to which the current model is related. + +For example, a `User` model could have a one-to-one field towards a `Profile` model. In such case, the `User` model could only have one associated `Profile` record, and the reverse would be true as well (a `Profile` record could only have one associated `User` record). In fact, a one-to-one field is really similar to a many-to-one field, but with an additional unicity constraint: + +```crystal +class Profile < Marten::Model + # ... +end + +class User < Marten::Model + # ... + field :profile, :one_to_one, to: Profile +end +``` + +In addition to the [common field options](#common-field-options), such fields support the following arguments: + +#### `to` + +The `to` argument **is required** and allows to specify the model class that is related to the model where the `one_to_one` field is defined. + +#### `related` + +The `related` argument allows defining the name of the reverse (or backward) relation on the targetted model. If we consider the previous example, it could be possible to define a `user` backward relation in order to let `Profile` records expose their related `User` record: + +```crystal +class Profile < Marten::Model + # ... +end + +class User < Marten::Model + # ... + field :profile, :one_to_one, to: Profile, related: :user +end +``` + +When the `related` argument is used, a method will be automatically created on the targetted model by using the chosen argument's value. For example, this means that the `User` record associated with a specific `Profile` record could be accessed through the use of the `Profile#user` method in the previous snippet. + +The default value is `nil`, which means that no reverse relation is defined on the targetted model by default. + +#### `on_delete` + +Same as [the similar option for the `#many_to_one` field](#on_delete). diff --git a/docs/versioned_docs/version-0.3/models-and-databases/reference/migration-operations.md b/docs/versioned_docs/version-0.3/models-and-databases/reference/migration-operations.md new file mode 100644 index 000000000..3a774db1e --- /dev/null +++ b/docs/versioned_docs/version-0.3/models-and-databases/reference/migration-operations.md @@ -0,0 +1,155 @@ +--- +title: Migration operations +description: Migration operations reference. +--- + +This page provides a reference for all the available migration operations that can be leveraged when writing migrations. + +## `add_column` + +The `add_column` operation allows adding a column to an existing table. It must be called with a table name as first argument, followed by a column definition (column name and attributes). + +For example: + +```crystal +add_column :test_table, :foo, :string, max_size: 255 +add_column :test_table, :new_id, :reference, to_table: :target_table, to_column: :id +``` + +## `add_index` + +The `add_index` operation allows adding an index to an existing table. It must be called with a table name as first argument, followed by an index definition (index name and indexed column names). + +For example: + +```crystal +add_index :test_table, :test_index, [:foo, :bar] +``` + +## `add_unique_constraint` + +The `add_unique_constraint` operation allows adding a unique constraint to an existing table. It must be called with a table name as first argument, followed by a unique constraint definition (constraint name and targetted column names). + +For example: + +```crystal +add_unique_constraint :test_table, :test_constraint, [:foo, :bar] +``` + +## `change_column` + +The `change_column` operation allows altering an existing column definition. It must be called with a table name as first argument, followed by a column definition (column name and attributes). + +For example: + +```crystal +change_column :test_table, :test_column, :string, max_size: 155, null: true +``` + +## `create_table` + +The `create_table` operation allows creating a new table, which includes the underlying column definitions, indexes, and unique constraints. It must be called with a table name as first argument and requires a block where columns, indexes, and unique constraints are defined. + +For example: + +```crystal +create_table :test_table do + column :id, :big_int, primary_key: true, auto: true + column :foo, :int, null: true + column :bar, :int, null: true + + unique_constraint :cname, [:foo, :bar] + index :index_name, [:foo, :bar] +end +``` + +## `delete_table` + +The `delete_table` operation allows deleting an existing table. It must be called with a table name as first argument. + +For example: + +```crystal +delete_table :test_table +``` + +## `execute` + +The `execute` operation allows executing custom SQL statements as part of a migration. It must be called with a forward statement as first positional argument, and it can also take a second positional argument in order to specify the statement to execute when unapplying the migration. + +For example: + +```crystal +execute( + ( + <<-SQL + SELECT 1 + SQL + ), + ( + <<-SQL + SELECT 2 + SQL + ) +) +``` + +## `remove_column` + +The `remove_column` operation allows removing an existing column from a table. It must be called with a table name as first argument, followed by a column name. + +For example: + +```crystal +remove_column :test_table, :test_column +``` + +## `remove_index` + +The `remove_index` operation allows removing an existing index from a table. It must be called with a table name as first argument, followed by an index name. + +For example: + +```crystal +remove_index :test_table, :test_index +``` + +## `remove_unique_constraint` + +The `remove_unique_constraint` operation allows removing an existing unique constraint from a table. It must be called with a table name as first argument, followed by a unique constraint name. + +For example: + +```crystal +remove_unique_constraint :test_table, :test_constraint +``` + +## `rename_column` + +The `rename_column` operation allows renaming an existing column in a table. It must be called with a table name as first argument, followed by the old column name, and the new one. + +For example: + +```crystal +rename_column :test_table, :old_column, :new_column +``` + +## `rename_table` + +The `rename_table` operation allows renaming an existing table. It must be called with the existing table name as first argument, followed by the new table name. + +For example: + +```crystal +rename_table :old_table, :new_table +``` + +## `run_code` + +The `run_code` operation allows to define that arbitrary methods will be called when applying and unapplying a migration. It must be called with a method name as first positional argument (the method that will be called when applying the migration), and it can also take an additional argument in order to specify the name of the method to execute when unapplying the migration. + +For example: + +```crystal +run_code :run_forward_code, :run_backward_code +``` diff --git a/docs/versioned_docs/version-0.3/models-and-databases/reference/query-set.md b/docs/versioned_docs/version-0.3/models-and-databases/reference/query-set.md new file mode 100644 index 000000000..27c3836c8 --- /dev/null +++ b/docs/versioned_docs/version-0.3/models-and-databases/reference/query-set.md @@ -0,0 +1,673 @@ +--- +title: Query set +description: Query set reference. +--- + +This page provides a reference for all the query set methods and available predicates that can be leveraged when filtering model records. + +## Query set laziness + +Query sets are **lazily evaluated**: defining a query set will usually not involve any database operations. Additionally, most methods provided by query sets also return new query set objects. Query sets are only translated to SQL queries hitting the underlying database when records need to be extracted or manipulated by the considered codebase. + +For example: + +```crystal +qset = Article.filter(title__startswith: "Top") # the query set is not evaluated +qset = qset.filter(author__first_name: "John") # the query set is not evaluated +puts qset # the query set is evaluated +``` + +In the above example the two filters are simply chained without these resulting in database hits. The query set is only evaluated when the actual records need to be printed. + +Overall, query sets are evaluated in the following situations: + +* when iterating over the underlying records (eg. when using `#each`) + + ```crystal + Article.filter(title__startswith: "Top").each do |article| + puts article + end + ``` + +* when retrieving the records for a specific range + + ```crystal + Article.filter(title__startswith: "Top")[4..10] + ``` + +* when printing the query set object (eg. using `puts`) + +## Methods that return new query sets + +Query sets provide a set of methods that allow to generate other (possibly filtered) query sets. Calling these methods won't result in the query set to be evaluated. + +### `[](range)` + +Returns the records corresponding to the passed range. + +If no records match the passed range, an `IndexError` exception is raised. If the current query set was already evaluated (records were retrieved from the database), an array of records will be returned. Otherwise, another sliced query set will be returned: + +```crystal +qset_1 = Article.all +qset_1.each { } +qset_1[2..6] # returns an array of Article records + +qset_2 = Article.all +qset_2[2..6] # returns a "sliced" query set +``` + +### `[]?(range)` + +Returns the records corresponding to the passed range. + +`nil` is returned if no records match the passed range. If the current query set was already evaluated (records were retrieved from the database), an array of records will be returned. Otherwise, another sliced query set will be returned: + +```crystal +qset_1 = Article.all +qset_1.each { } +qset_1[2..6]? # returns an array of Article records + +qset_2 = Article.all +qset_2[2..6]? # returns a "sliced" query set +``` + +### `all` + +Allows retrieving all the records of a specific model. `#all` can be used as a class method from any model class, or it can be used as an instance method from any query set object. In this last case, calling `#all` returns a copy of the current query set. + +For example: + +```crystal +qset = Article.all # returns a query set matching "all" the records of the Article model +qset2 = qset.all # returns a copy of the initial query set +``` + +### `distinct` + +Returns a new query set that will use `SELECT DISTINCT` or `SELECT DISTINCT ON` in its SQL query. + +If you use this method without arguments, a `SELECT DISTINCT` statement will be used at the database level. If you pass field names as arguments, a `SELECT DISTINCT ON` statement will be used to eliminate any duplicated rows based on the specified fields: + +```crystal +query_set_1 = Post.all.distinct +query_set_2 = Post.all.distinct(:title) +``` + +It should be noted that it is also possible to follow associations of direct related models too by using the [double underscores notation](../queries#joins-and-filtering-relations) (`__`). For example the following query will select distinct records based on a joined "author" attribute: + +``` +query_set = Post.all.distinct(:author__name) +``` + +Finally, it should be noted that `#distinct` cannot be used on [sliced query sets](#range). + +### `exclude` + +Returns a query set whose records do not match the given set of filters. + +The filters passed to this method method can be specified using the [standard predicate format](../queries#basic-querying-capabilities). If multiple filters are specified, they will be joined using an **AND** operator at the SQL level: + +```crystal +query_set = Post.all +query_set.exclude(title: "Test") +query_set.exclude(title__startswith: "A") +``` + +Complex filters can also be used as part of this method by leveraging [`q` expressions](../queries#complex-filters-with-q-expressions): + +```crystal +query_set = Post.all +query_set.exclude { (q(name: "Foo") | q(name: "Bar")) & q(is_published: True) } +``` + +### `filter` + +Returns a query set matching a specific set of filters. + +The filters passed to this method method can be specified using the [standard predicate format](../queries#basic-querying-capabilities). If multiple filters are specified, they will be joined using an **AND** operator at the SQL level: + +```crystal +query_set = Post.all +query_set.filter(title: "Test") +query_set.filter(title__startswith: "A") +``` + +Complex filters can also be used as part of this method by leveraging [`q` expressions](../queries#complex-filters-with-q-expressions): + +```crystal +query_set = Post.all +query_set.filter { (q(name: "Foo") | q(name: "Bar")) & q(is_published: True) } +``` + +### `join` + +Returns a queryset whose specified `relations` are "followed" and joined to each result (see [Queries](../queries#joins-and-filtering-relations) for an introduction about this capability). + +When using `#join`, the specified foreign-key relationships will be followed and each record returned by the queryset will have the corresponding related objects already selected and populated. Using `#join` can result in performance improvements since it can help reduce the number of SQL queries, as illustrated by the following example: + +```crystal +query_set = Post.all + +p1 = query_set.get(id: 1) +puts p1.author # hits the database to retrieve the related "author" + +p2 = query_set.join(:author).get(id: 1) +puts p2.author # doesn't hit the database since the related "author" was already selected +``` + +It should be noted that it is also possible to follow foreign keys of direct related models too by using the double underscores notation (`__`). For example the following query will select the joined "author" and its associated "profile": + +```crystal +query_set = Post.all +query_set.join(:author__profile) +``` + +### `none` + +Returns a query set that will always return an empty array of records, without querying the database. + +Once this method is used, any subsequent method calls (such as extra filters) will continue returning an empty array of records: + +```crystal +query_set = Post.all +query_set.none.exists? # => false +``` + +### `order` + +Allows specifying the ordering in which records should be returned when evaluating the query set. + +Multiple fields can be specified in order to define the final ordering. For example: + +```crystal +query_set = Post.all +query_set.order("-published_at", "title") +``` + +In the above example, records would be ordered by descending publication date (because of the `-` prefix), and then by title (ascending). + +### `raw` + +Returns a raw query set for the passed SQL query and optional parameters. + +This method returns a [`Marten::DB::Query::RawSet`](pathname:///api/0.3/Marten/DB/Query/RawSet.html) object, which allows to iterate over the model records matched by the passed SQL query. For example: + +```crystal +Article.all.raw("SELECT * FROM articles") +``` + +Additional parameters can also be specified if the query needs to be parameterized. Those can be specified as positional or named arguments. For example: + +```crystal +# Using splat positional parameters: +Article.all.raw("SELECT * FROM articles WHERE title = ? and created_at > ?", "Hello World!", "2022-10-30") + +# Using an array of positional parameters: +Article.all.raw("SELECT * FROM articles WHERE title = ? and created_at > ?", ["Hello World!", "2022-10-30"]) + +# Using double splat named parameters: +Article.all.raw( + "SELECT * FROM articles WHERE title = :title and created_at > :created_at", + title: "Hello World!", + created_at: "2022-10-30" +) + +# Using a hash of named parameters: +Article.all.raw( + "SELECT * FROM articles WHERE title = :title and created_at > :created_at", + { + title: "Hello World!", + created_at: "2022-10-30", + } +) +``` + +Please refer to [Raw SQL](../raw-sql) to learn more about performing raw SQL queries. + +### `reverse` + +Allows reversing the order of the current query set. + +For example, this would return all the `Article` records ordered by descending title: + +```crystal +query_set = Article.all.order(:title) +query_set.reverse +``` + +### `using` + +Allows defining which database alias should be used when evaluating the query set. + +For example: + +```crystal +query_set_1 = Article.all.filter(published: true) # records are retrieved from the default database +query_set_2 = Article.all.filter(published: true).using(:other) # records are retrieved from the "other" database +``` + +The value passed to `#using` must be a valid database alias that was used to configure an additional database as part of the [database settings](../../development/reference/settings#database-settings). + +## Methods that do not return new query sets + +Query sets also provide a set of methods that will usually result in specific SQL queries to be executed in order to return values that don't correspond to new query sets. + +### `count` + +Returns the number of records that are targeted by the current query set. + +For example: + +```crystal +Article.all.count # returns the number of article records +Article.filter(title__startswith: "Top").count # returns the number of articles whose title start with "Top" +``` + +Note that this method will trigger a `SELECT COUNT` SQL query if the query set was not already evaluated: when this happens, no model records will be instantiated since the records count will be determined at the database level. If the query set was already evaluated, the underlying array of records will be used to return the records count instead of running a dedicated SQL query. + +### `create` + +Creates a model instance and saves it to the database if it is valid. + +The new model instance is initialized by using the attributes defined in the passed double splat argument. Regardless of whether it is valid or not (and thus persisted to the database or not), the initialized model instance is returned by this method: + +```crystal +query_set = Post.all +query_set.create(title: "My blog post") +``` + +This method can also be called with a block that is executed for the new object. This block can be used to directly initialize the object before it is persisted to the database: + +```crystal +query_set = Post.all +query_set.create(title: "My blog post") do |post| + post.complex_attribute = compute_complex_attribute +end +``` + +### `create!` + +Creates a model instance and saves it to the database if it is valid. + +The model instance is initialized using the attributes defined in the passed double splat argument. If the model instance is valid, it is persisted to the database ; otherwise a `Marten::DB::Errors::InvalidRecord` exception is raised. + +```crystal +query_set = Post.all +query_set.create!(title: "My blog post") +``` + +This method can also be called with a block that is executed for the new object. This block can be used to directly initialize the object before it is persisted to the database: + +```crystal +query_set = Post.all +query_set.create!(title: "My blog post") do |post| + post.complex_attribute = compute_complex_attribute +end +``` + +### `delete` + +Deletes the records corresponding to the current query set and returns the number of deleted records. + +By default, related objects will be deleted by following the [deletion strategy](./fields#on_delete) defined in each foreign key field if applicable, unless the `raw` argument is set to `true`. When the `raw` argument is set to `true`, a raw SQL delete statement will be used to delete all the records matching the currently applied filters. Note that using this option could cause errors if the underlying database enforces referential integrity. + +```crystal +Article.all.delete # deletes all the Article records +Article.filter(title__startswith: "Top").delete # deletes all the articles whose title start with "Top" +``` + +### `each` + +Allows iterating over the records that are targeted by the current query set. + +This method can be used to define a block that iterates over the records that are targeted by a query set: + +```crystal +Post.all.each do |post| + # Do something with the post +end +``` + +### `exists?` + +Returns `true` if the current query set matches at least one record, or `false` otherwise. + +```crystal +Article.filter(title__startswith: "Top").exists? +``` + +Note that this method will trigger a very simple `SELECT EXISTS` SQL query if the query set was not already evaluated: when this happens, no model records will be instantiated since the records existence will be determined at the database level. If the query set was already evaluated, the underlying array of records will be used to determine if records exist or not. + +It should be noted that `#exists?` can also take additional filters or `q()` expressions as arguments. This allows to apply additional filters to the considered query set in order to perform the check. For example: + +```crystal +query_set = Tag.filter(name__startswith: "c") +query_set.exists?(is_active: true) +query_set.exists? { q(is_active: true) } +``` + +### `first` + +Returns the first record that is matched by the query set, or `nil` if no records are found. + +```crystal +Article.first +Article.filter(title__startswith: "Top").first +``` + +### `first!` + +Returns the first record that is matched by the query set, or raises a `NilAssertionError` exception if no records are found. + +```crystal +Article.first! +Article.filter(title__startswith: "Top").first! +``` + +### `get` + +Returns the model instance matching the given set of filters. + +Model fields such as primary keys or fields with a unique constraint should be used here in order to retrieve a specific record: + +```crystal +query_set = Post.all +post_1 = query_set.get(id: 123) +post_2 = query_set.get(id: 456, is_published: false) +``` + +Complex filters can also be used as part of this method by leveraging [`q` expressions](../queries#complex-filters-with-q-expressions): + +```crystal +query_set = Post.all +post_1 = query_set.get { q(id: 123) } +post_2 = query_set.get { q(id: 456, is_published: false) } +``` + +If the specified set of filters doesn't match any records, the returned value will be `nil`. Moreover, in order to ensure data consistency this method will raise a `Marten::DB::Errors::MultipleRecordsFound` exception if multiple records match the specified set of filters. + +### `get!` + +Returns the model instance matching the given set of filters. + +Model fields such as primary keys or fields with a unique constraint should be used here in order to retrieve a specific record: + +```crystal +query_set = Post.all +post_1 = query_set.get!(id: 123) +post_2 = query_set.get!(id: 456, is_published: false) +``` + +Complex filters can also be used as part of this method by leveraging [`q` expressions](../queries#complex-filters-with-q-expressions): + +```crystal +query_set = Post.all +post_1 = query_set.get! { q(id: 123) } +post_2 = query_set.get! { q(id: 456, is_published: false) } +``` + +If the specified set of filters doesn't match any records, a `Marten::DB::Errors::RecordNotFound` exception will be raised. Moreover, in order to ensure data consistency this method will raise a `Marten::DB::Errors::MultipleRecordsFound` exception if multiple records match the specified set of filters. + +### `get_or_create` + +Returns the model record matching the given set of filters or create a new one if no one is found. + +Model fields that uniquely identify a record should be used here. For example: + +```crystal +tag = Tag.all.get_or_create(label: "crystal") +``` + +When no record is found, the new model instance is initialized by using the attributes defined in the double splat arguments. Regardless of whether it is valid or not (and thus persisted to the database or not), the initialized model instance is returned by this method. + +This method can also be called with a block that is executed for new objects. This block can be used to directly initialize new records before they are persisted to the database: + +```crystal +tag = Tag.all.get_or_create(label: "crystal") do |new_tag| + new_tag.active = false +end +``` + +In order to ensure data consistency, this method will raise a `Marten::DB::Errors::MultipleRecordsFound` exception if multiple records match the specified set of filters. + +### `get_or_create!` + +Returns the model record matching the given set of filters or create a new one if no one is found. + +Model fields that uniquely identify a record should be used here. For example: + +```crystall +tag = Tag.all.get_or_create!(label: "crystal") +``` + +When no record is found, the new model instance is initialized by using the attributes defined in the double splat arguments. If the new model instance is valid, it is persisted to the database ; otherwise a `Marten::DB::Errors::InvalidRecord` exception is raised. + +This method can also be called with a block that is executed for new objects. This block can be used to directly initialize new records before they are persisted to the database: + +```crystal +tag = Tag.all.get_or_create!(label: "crystal") do |new_tag| + new_tag.active = false +end +``` + +In order to ensure data consistency, this method will raise a `Marten::DB::Errors::MultipleRecordsFound` exception if multiple records match the specified set of filters. + +### `includes?` + +Returns `true` if a specific model record is included in the query set. + +This method can be used to verify the membership of a specific model record in a given query set. If the query set is not evaluated yet, a dedicated SQL query will be executed in order to perform this check (without loading the entire list of records that are targeted by the query set). This is especially interesting for large query sets where we don't want all the records to be loaded in memory in order to perform such check. + +```crystal +tag = Tag.get!(name: "crystal") +query_set = Tag.filter(name__startswith: "c") +query_set.includes?(tag) # => true +``` + +### `last` + +Returns the last record that is matched by the query set, or `nil` if no records are found. + +```crystal +Article.last +Article.filter(title__startswith: "Top").last +``` + +### `last!` + +Returns the last record that is matched by the query set, or raises a `NilAssertionError` exception if no records are found. + +```crystal +Article.last! +Article.filter(title__startswith: "Top").last! +``` + +### `paginator` + +Returns a paginator that can be used to paginate the current query set. + +This method returns a [`Marten::DB::Query::Paginator`](pathname:///Marten/DB/Query/Paginator.html) object, which can then be used to retrieve specific pages. A page size must be specified when calling this method. + +For example: + +```crystal +query_set = Article.all +paginator = query_set.paginator(10) +paginator.page(1) # Returns the first page of records +``` + +### `pick` + +Returns specific column values for a single record without actually loading it. + +This method allows to easily select specific column values for a single record from the current query set. This allows retrieving specific column values without actually loading the entire record, and as such this is most useful for query sets that have been narrowed down to match a single record. The method returns an array containing the requested column values, or `nil` if no record was matched by the current query set. + +For example: + +```crystal +Post.filter(pk: 1).pick("title", "published") +# => ["First article", true] +``` + +### `pick!` + +Returns specific column values for a single record without actually loading it. + +This method allows to easily select specific column values for a single record from the current query set. This allows retrieving specific column values without actually loading the entire record, and as such this is most useful for query sets that have been narrowed down to match a single record. The method returns an array containing the requested column values, or raises `NilAssertionError` if no record was matched by the current query set. + +For example: + +```crystal +Post.filter(pk: 1).pick!("title", "published") +# => ["First article", true] +``` + +### `pluck` + +Returns specific column values without loading entire record objects. + +This method allows to easily select specific column values from the current query set. This allows retrieving specific column values without actually loading entire records. The method returns an array containing one array with the actual column values for each record targeted by the query set. + +For example: + +```crystal +Post.all.pluck("title", "published") +# => [["First article", true], ["Upcoming article", false]] +``` + +### `size` + +Alias for [`#count`](#count): returns the number of records that are targetted by the query set. + +### `update` + +Updates all the records matched by the current query set with the passed values. + +This method allows to update all the records that are matched by the current query set with the values defined in the passed double splat argument. It returns the number of records that were updated: + +```crystal +query_set = Post.all +query_set.update(title: "Updated") # => 42 +``` + +It should be noted that this method results in a regular `UPDATE` SQL statement. As such, the records that are updated through the use of this method won't be instantiated nor validated, and no callbacks will be executed for them either. + +## Field predicates + +Below are listed all the available [field predicates](../queries#field-predicates) that can be used when filtering query sets. + +### `contains` + +Allows filtering records based on field values that contain a specific substring. Note that this is a **case-sensitive** predicate. + +```crystal +Article.all.filter(title__contains: "tech") +``` + +### `endswith` + +Allows filtering records based on field values that end with a specific substring. Note that this is a **case-sensitive** predicate. + +```crystal +Article.all.filter(title__endswith: "travel") +``` + +### `exact` + +Allows filtering records based on a specific field value (exact match). Note that providing a `nil` value will result in a `IS NULL` check at the SQL level. + +This is the default predicate; as such it is not necessary to specify it when filtering records. The following two query sets are equivalent: + +```crystal +Article.all.filter(published: true) +Article.all.filter(published__exact: true) +``` + +### `gte` + +Allows filtering records based on field values that are greater than or equal to a specified value. + +```crystal +Article.all.filter(rating__gte: 10) +``` + +### `gt` + +Allows filtering records based on field values that are greater than a specified value. + +```crystal +Article.all.filter(rating__gt: 10) +``` + +### `icontains` + +Allows filtering records based on field values that contain a specific substring, in a case-insensitive way. + +```crystal +Article.all.filter(title__icontains: "tech") +``` + +### `iendswith` + +Allows filtering records based on field values that end with a specific substring, in a case-insensitive way. + +```crystal +Article.all.filter(title__iendswith: "travel") +``` + +### `iexact` + +Allows filtering records based on a specific field value (exact match), in a case-insensitive way. + +```crystal +Article.all.filter(title__iexact: "Top blog posts") +``` + +### `istartswith` + +Allows filtering records based on field values that start with a specific substring, in a case-insensitive way. + +```crystal +Article.all.filter(title__istartswith: "top") +``` + +### `in` + +Allows filtering records based on field values that are contained in a specific array of values. + +```crystal +Tag.all.filter(slug__in=["foo", "bar", "xyz"]) +``` + +### `isnull` + +Allows filtering records based on field values that should be null or not null. + +```crystal +Article.all.filter(subtitle__isnull: true) +Article.all.filter(subtitle__isnull: false) +``` + +### `lte` + +Allows filtering records based on field values that are less than or equal to a specified value. + +```crystal +Article.all.filter(rating__lte: 10) +``` + +### `lt` + +Allows filtering records based on field values that are less than a specified value. + +```crystal +Article.all.filter(rating__lt: 10) +``` + +### `startswith` + +Allows filtering records based on field values that start with a specific substring. Note that this is a **case-sensitive** predicate. + +```crystal +Article.all.filter(title__startswith: "Top") +``` diff --git a/docs/versioned_docs/version-0.3/models-and-databases/reference/table-options.md b/docs/versioned_docs/version-0.3/models-and-databases/reference/table-options.md new file mode 100644 index 000000000..cf40db364 --- /dev/null +++ b/docs/versioned_docs/version-0.3/models-and-databases/reference/table-options.md @@ -0,0 +1,57 @@ +--- +title: Table options +description: Table options reference. +--- + +This page provides a reference for all the table options that can be leveraged when defining models. + +## Table name + +Table names for models are automatically generated from the model name and the label of the associated application. That being said, it is possible to specifically override the name of a model table by leveraging the [`#db_table`](pathname:///api/0.3/Marten/DB/Model/Table/ClassMethods.html#db_table(db_table%3AString|Symbol)-instance-method) class method, which requires a table name string or symbol. + +For example: + +```crystal +class Article < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :title, :string, max_size: 255 + field :content, :text + +// highlight-next-line + db_table :articles +end +``` + +## Table indexes + +Multifields indexes can be configured in a model by leveraging the [`#db_index`](pathname:///api/0.3/Marten/DB/Model/Table/ClassMethods.html#db_index(name%3AString|Symbol%2Cfield_names%3AArray(String)|Array(Symbol))%3ANil-instance-method) class method. This method requires an index name argument as well as an array of targeted field names. + +For example: + +```crystal +class Person < Marten::Model + field :id, :int, primary_key: true, auto: true + field :first_name, :string, max_size: 50 + field :last_name, :string, max_size: 50 + +// highlight-next-line + db_index :person_full_name_index, field_names: [:first_name, :last_name] +end +``` + +## Table unique constraints + +Multifields unique constraints can be configured in a model by leveraging the [`#db_unique_constraint`](pathname:///api/0.3/Marten/DB/Model/Table/ClassMethods.html#db_unique_constraint(name%3AString|Symbol%2Cfield_names%3AArray(String)|Array(Symbol))%3ANil-instance-method) class method. This method requires an index name argument as well as an array of targeted field names. + +For example: + +```crystal +class Booking < Marten::Model + field :id, :int, primary_key: true, auto: true + field :room, :string, max_size: 50 + field :date, :date, max_size: 50 + +// highlight-next-line + db_unique_constraint :booking_room_date_constraint, field_names: [:room, :date] +end +``` diff --git a/docs/versioned_docs/version-0.3/models-and-databases/transactions.md b/docs/versioned_docs/version-0.3/models-and-databases/transactions.md new file mode 100644 index 000000000..981e06509 --- /dev/null +++ b/docs/versioned_docs/version-0.3/models-and-databases/transactions.md @@ -0,0 +1,65 @@ +--- +title: Database transactions +description: Learn how to leverage database transactions. +sidebar_label: Transactions +--- + +Transactions are blocks whose underlying SQL statements are committed to the database as one atomic action only if they can complete without errors. Marten provides a few mechanisms to control how database transactions are performed and managed. + +## The basics + +Transactions are essential in order to enforce database integrity. Whenever you are in a situation where you have more than one SQL operations that must be executed together or not at all, then you should consider wrapping all these operations in a dedicated transaction. Transaction blocks can be created by leveraging the `#transaction` method, which can be called either on [model records](pathname:///api/0.3/Marten/DB/Model/Connection.html#transaction(using%3ANil|String|Symbol%3Dnil%2C%26block)-instance-method) or [model classes](pathname:///api/0.3/Marten/DB/Model/Connection/ClassMethods.html#transaction(using%3ANil|String|Symbol%3Dnil%2C%26)-instance-method). + +For example: + +```crystal +MyModel.transaction do + my_record.save! + my_other_record.save! +end +``` + +With the above snippet, both records will be saved _only_ if each save operation completes successfully (that is if no exception is raised). If an exception occurs as part of one of the save operations (eg. if one of the records is invalid), then no records will be saved. + +It should be noted that there is no difference between calling `#transaction` on [a model record](pathname:///api/0.3/Marten/DB/Model/Connection.html#transaction(using%3ANil|String|Symbol%3Dnil%2C%26block)-instance-method) or [a model class](pathname:///api/0.3/Marten/DB/Model/Connection/ClassMethods.html#transaction(using%3ANil|String|Symbol%3Dnil%2C%26)-instance-method). It's also worth mentioning that the models manipulated within a transaction block that result in SQL statements can be of different classes. For example, the following two transactions would be equivalent: + +```crystal +MyModel.transaction do + MyModel.create!(foo: "bar") + MyOtherModel.create!(foo: "bar") +end + +MyOtherModel.transaction do + MyModel.create!(foo: "bar") + MyOtherModel.create!(foo: "bar") +end +``` + +:::info +When transaction blocks are nested, this results in all the database statements of the inner transaction to be added to the outer transaction. As such, there is only one "effective" transaction at any given time when transaction blocks are nested. +::: + +## Automatic transactions + +Basic model operations such as [creating](./introduction#create), [updating](./introduction#update), or [deleting](./introduction#delete) records are automatically wrapped in a transaction. This helps in ensuring that any exception that is raised in the context of validations or as part of `after_*` [callbacks](./callbacks) (ie. `after_create`, `after_update`, `after_save`, and `after_delete`) will also roll back the current transaction. + +The consequence of this is that the changes you make to the database in these callbacks will not be "visible" until the transaction is complete. For example, this means that if you are triggering something (like an asynchronous job) that needs to leverage the changes introduced by a model operation, then you should probably not use the regular `after_*` callbacks. Instead, you should leverage [`after_commit`](./callbacks#aftercommit) callbacks (which are the only callbacks that are triggered _after_ a model operation has been committed to the database). + +## Exception handling and rollbacks + +As mentioned before, any exception that is raised from within a transaction block will result in the considered transaction being rolled back. Moreover, it should be noted that raised exceptions will also be propagated outside of the transaction block, which means that your codebase should catch these accordingly if applicable. + +If you need to roll back a transaction _manually_ from within a transaction itself while ensuring that no exception is propagated outside of the block, then you can make use of the [`Marten::DB::Errors::Rollback`](pathname:///api/0.3/Marten/DB/Errors/Rollback.html) exception: when this specific exception is raised from inside a transaction block, the transaction will be rolled back and the transaction block will return `false`. + +For example: + +```crystal +transaction_committed = MyModel.transaction do + MyModel.create!(foo: "bar") + MyOtherModel.create!(foo: "bar") + + raise Marten::DB::Errors::Rollback.new("Stop!") if should_rollback? +end + +transaction_committed # => false +``` diff --git a/docs/versioned_docs/version-0.3/models-and-databases/validations.md b/docs/versioned_docs/version-0.3/models-and-databases/validations.md new file mode 100644 index 000000000..328e86997 --- /dev/null +++ b/docs/versioned_docs/version-0.3/models-and-databases/validations.md @@ -0,0 +1,170 @@ +--- +title: Model validations +description: Learn how to validate model records. +sidebar_label: Validations +--- + +Model instances _should_ be validated before being persisted to the database. As such models provide a convenient way to define validation rules through the use of model fields and through the use of a custom validation rules DSL. The underlying validation logics are completely database-agnostic, cannot be skipped (unless explicitly specified), and can be unit-tested easily. + +Validation rules can be inherited from the fields in your model depending on the options you used and the type of your fields (for example fields using `blank: false` will make the associated record validation fail if the field value is blank). They can also be explicitly specified in your model class, which is useful if you need to implement custom validation logics. + +## Overview + +### A short example + +Let's consider the following example: + +```crystal +class User < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :name, :string, max_size: 128, blank: false +end +``` + +In the above snippet, a `User` model is defined and it is specified that the `name` field must be present (`blank: false`) and that the associated value cannot exceed 128 characters. + +Given these characteristics, it is possible to create `User` instances and to validate them through the use of the `#valid?` method: + +```crystal +user_1 = User.new +user_1.valid? # => false + +user_2 = User.new(name: "0" * 200) +user_2.valid? # => false + +user_3 = User.new(name: "John Doe") +user_3.valid? # => true +``` + +As you can see in the above examples, the first two users are invalid because either the `name` is not specified or because its value exceeds the maximum characters limit. The last user is valid though because it has a `name` that is less than 128 characters. + +### When does model validation happen? + +Model instances are validated when they are created or updated, before any values are persisted to the database. Methods like `#create` or `#save` automatically run validations. They return `false` to indicate that the considered object is invalid (and they return `true` if the object is valid). The `#create` and `#save` methods also have bang counterparts (`#create!` and `#save!`) that will explicitly raise a validation error (instance of `Marten::DB::Errors::InvalidRecord`) in case of invalid records. + +For example: + +```crystal +user = User.new +user.save +# => false +user.save! +# => Unhandled exception: Record is invalid (Marten::DB::Errors::InvalidRecord) +``` + +When validating model records, the validation rules that are inherited by fields will be executed first and then any custom validation rule defined in the model will be applied. + +### Running model validations + +As mentioned previously, validation rules will be executed automatically when calling the `#create` or `#save` methods on a model record. It is also possible to manually verify whether a model instance is valid or not using the `#valid?` and `#invalid?` methods: + +```crystal +user = User.new +user.valid? # => false +user.invalid? # => true +``` + +## Field validation rules + +As mentioned previously, fields can contribute validation rules to your models. These validation rules can be inherited: + +* from the field type itself: some fields will validate that values are of a specific type (for example a `uuid` field will not validate values that don't correspond to valid UUIDs) +* from the field options you define (for example fields using `blank: true` won't accept empty values) + +Please refer to the [fields reference](./reference/fields) in order to learn more about the supported field types and their associated options. + +## Custom validation rules + +Custom validate rules can be defined through the use of the `#validate` macro. This macro lets you configure the name of a validation method that should be called when a model instance is validated. Inside this method, you can implement any validation logic that you might require and add errors to your model instances if they are identified as invalid. + +For example: + +```crystal +class User < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :name, :string, max_size: 128, blank: false + + validate :validate_name_is_not_forbidden + + private def validate_name_is_not_forbidden + errors.add(:name, "admin can't be used!") if name == "admin" + end +end +``` + +In the above snippet, a custom validation method ensures that the `name` of a `User` model instance can't be set to `"admin"`: if the name is set to `"admin"`, then a specific error (associated with the `name` attribute) is added to the model instance (which makes it invalid). + +## Validation errors + +Methods like `#valid?` or `#invalid?` only let you know whether a model instance is valid or invalid. But you'll likely want to know exactly what are the actual errors or how to add new ones. + +As such, every model instance has an associated error set, which is an instance of [`Marten::Core::Validation::ErrorSet`](pathname:///api/0.3/Marten/Core/Validation/ErrorSet.html). + +### Inspecting errors + +A model instance error set lets you access all the errors of a specific model instance. For example: + +```crystal +user = User.new + +user.valid? +# => false + +user.errors.size +# => 1 + +user.errors +# => #]> +``` + +As you can see, the error set gives you the ability to know how many errors are affecting your model instance. Each error provides some additional information as well: + +* the associated field name (which can be `nil` if the error is global) +* the error message +* the error type, which is optional (`blank` in the previous example) + +You can also access the errors that are associated with a specific field very easily by using the `#[]` method: + +```crystal +user.errors[:name] +# => [#] +``` + +Global errors (errors affecting the whole model instances or multiple fields at once) can be listed through the use of the `#global` method. + +### Adding errors + +Errors can be added to an error set through the use of the `#add` method. This method takes a field name, a message, and an optional error type: + +```crystal +user.errors.add(:name, "Name is invalid") # error type is "invalid" +user.errors.add(:name, "Name is invalid", type: :invalid_name) # error type is "invalid_name" +``` + +Global errors can be specified through the use of an alternative `#add` method that doesn't take a field name: + +```crystal +user.errors.add("User is invalid") # error type is "invalid" +user.errors.add("User is invalid", type: :invalid_user) # error type is "invalid_user" +``` + +## Skipping validations + +Model validations can be explicitly skipped when using the `#save` or `#save!` methods. To do so, the `validate: false` argument can be used: + +```crystal +user = User.new +user.save(validate: false) +``` + +:::caution +It is generally not a good idea to skip validation that way. This technique should be used with caution! +::: diff --git a/docs/versioned_docs/version-0.3/prologue.md b/docs/versioned_docs/version-0.3/prologue.md new file mode 100644 index 000000000..9f9f1ed08 --- /dev/null +++ b/docs/versioned_docs/version-0.3/prologue.md @@ -0,0 +1,24 @@ +--- +slug: / +--- + +# Prologue + +**Marten** is a Crystal Web framework that enables pragmatic development and rapid prototyping. It provides a consistent and extensible set of tools that developers can leverage to build web applications without reinventing the wheel. + +## Getting started + +Are you new to the Marten web framework? The following resources will help you get started: + +* The [installation guide](./getting-started/installation.md) will help you install Crystal and the Marten CLI +* The [tutorial](./getting-started/tutorial.md) will help you discover the main features of the framework by creating a simple web application + +## Browsing the documentation + +The Marten documentation contains multiple pages and references that don't necessarily serve the same purpose. To help you browse this documentation, here is an overview of the different sorts of content you might encounter: + +* **Topic-specific guides** discuss the key concepts of the framework. They provide explanations and useful information about things like [models](./models-and-databases), [handlers](./handlers-and-http), and [templates](./templates) +* **Reference pages** provide a curated technical reference of the framework APIs +* **How-to guides** document how to solve common problems when working with the framework. Those can cover things like deployments, app development, etc + +Additionally, an automatically-generated [API reference](pathname:///api/0.3/index.html) is also available to dig into Marten's internals. diff --git a/docs/versioned_docs/version-0.3/schemas.mdx b/docs/versioned_docs/version-0.3/schemas.mdx new file mode 100644 index 000000000..256fca3bb --- /dev/null +++ b/docs/versioned_docs/version-0.3/schemas.mdx @@ -0,0 +1,34 @@ +--- +title: Schemas +--- + +import DocCard from '@theme/DocCard'; + +Schemas are classes that define how input data should be serialized / deserialized, and validated. Schemas are usually used when processing web requests containing form data or pre-defined payloads. + +## Guides + +
+
+ +
+
+ +
+
+ +## How-to's + +
+
+ +
+
+ +## Reference + +
+
+ +
+
diff --git a/docs/versioned_docs/version-0.3/schemas/how-to/create-custom-schema-fields.md b/docs/versioned_docs/version-0.3/schemas/how-to/create-custom-schema-fields.md new file mode 100644 index 000000000..86e0eb51a --- /dev/null +++ b/docs/versioned_docs/version-0.3/schemas/how-to/create-custom-schema-fields.md @@ -0,0 +1,201 @@ +--- +title: Create custom schema fields +description: How to create custom schema fields. +--- + +Marten gives you the ability to create your own custom schema field implementations. Those can involve custom validations, behaviors, and errors. You can leverage these custom fields as part of your project's schema definitions, and you can even distribute them to let other projects use them. + +## Schema fields: scope and responsibilities + +Schema fields have the following responsibilities: + +* they define how field values are deserialized and serialized +* they define how these values are validated + +When creating a custom schema field, there are usually two approaches that you can consider depending on the amount of customization that you want to implement. You can either: + +* leverage an existing built-in schema field (eg. integer, string, etc) and add custom behaviors and validations to it +* or create a new schema field from scratch + +## Registering new schema fields + +Regardless of the approach you take in order to define new schema field classes ([subclassing built-in fields](#subclassing-existing-schema-fields), or [creating new ones from scratch](#creating-new-schema-fields-from-scratch)), these classes must be registered to the Marten's global fields registry in order to make them available for use when defining schemas. + +To do so, you will have to call the [`Marten::Schema::Field#register`](pathname:///api/0.3/Marten/Schema/Field.html#register(id%2Cfield_klass)-macro) method with the identifier of the field you wish to use, and the actual field class. For example: + +```crystal +Marten::Schema::Field.register(:foo, FooField) +``` + +The identifier you pass to `#register` can be a symbol or a string. This is the identifier that is then made available to schema classes in order to define their fields: + +```crystal +class MySchema < Marten::Schema + field :title, :string + // highlight-next-line + field :test, :foo +end +``` + +The call to `#register` can be made from anywhere in your codebase, but obviously, you will want to ensure that it is done before requiring your schema classes: indeed, Marten will make the compilation of your project fail if it can't find the field type you are trying to use as part of a schema definition. + +## Subclassing existing schema fields + +This is probably the easiest way to create a custom field: if the field you want to create can be derived from one of the [built-in schema fields](../reference/fields) (usually those correspond to primitive types), then you can easily subclass the corresponding class and customize it so that it suits your needs. + +For example, implementing a custom "email" field could be done by subclassing the existing [`Marten::Schema::Field::String`](pathname:///api/0.3/Marten/Schema/Field/String.html) class. Indeed, an "email" field is essentially a string with a pre-defined maximum size and some additional validation logic: + +```crystal +class EmailField < Marten::Schema::Field::String + def initialize( + @id : ::String, + @required : ::Bool = true, + @max_size : ::Int32? = 254, + @min_size : ::Int32? = nil + ) + @strip = true + end + + def validate(schema, value) + return if !value.is_a?(::String) + + # Leverage string's built-in validations (max size, min size). + super + + if !EmailValidator.valid?(value) + schema.errors.add(id, "Provide a valid email address") + end + end +end +``` + +In the above snippet, the `EmailField` class simply overrides the `#validate` method so that it implements validation rules that are specific to the use case of email addresses (while also ensuring that regular string validations are executed as well). + +Everything that is described in the following section about [creating schema fields from scratch](#creating-new-schema-fields-from-scratch) also applies to the case of subclassing existing schema fields: the same methods can be overridden if necessary, but leveraging an existing class can save you some work. + +## Creating new schema fields from scratch + +Creating new schema fields from scratch involves subclassing the [`Marten::Schema::Field::Base`](pathname:///api/0.3/Marten/Schema/Field/Base.html) abstract class. Because of this, the new field class is required to implement a set of mandatory methods. These mandatory methods, and some other ones that are optional (but interesting in terms of capabilities), are described in the following sections. + +### Mandatory methods + +#### `deserialize` + +The `#deserialize` method is responsible for deserializing a schema field value. Indeed, the raw value of a schema field usually comes from a request's data and needs to be converted to another format. For example, a `uuid` field might need to convert a `String` value to a proper `UUID` object: + +```crystal +def deserialize(value) : ::UUID? + return if empty_value?(value) + + case value + when Nil + value + when ::String + value.empty? ? nil : ::UUID.new(value) + when JSON::Any + deserialize(value.raw) + else + raise_unexpected_field_value(value) + end +rescue ArgumentError + raise_unexpected_field_value(value) +end +``` + +Fields can be configured as required or not ([`required`](../reference/fields#required) option), this means that you will usually want to handle the case of `nil` values as part of this methods and return `nil` if the incoming value is `nil`. It should also be noted that incoming values can be any JSON data (`JSON::Any`), which means that you need to handle this case properly as well. + +If the value can't be processed properly by your field class, then it may be necessary to raise an exception. To do that you can leverage the `#raise_unexpected_field_value` method, which will raise a `Marten::Schema::Errors::UnexpectedFieldValue` exception. + +#### `serialize` + +The `#serialize` method is responsible for serializing a field value, which is essentially the reverse of the [`#deserialize`](#deserialize) method. As such, this method must convert a field value from the "Crystal" representation to the "raw" schema representation. + +For example, this method could return the string representation of a `UUID` object: + +```crystal +def serialize(value) : ::String? + value.try(&.to_s) +end +``` + +Again, if the value can't be processed properly by the field class, it may be necessary to raise an exception. To do that you can leverage the `#raise_unexpected_field_value` method, which will raise a `Marten::Schema::Errors::UnexpectedFieldValue` exception. + +### Other useful methods + +#### `initialize` + +The default `#initialize` method that is provided by the [`Marten::Schema::Field::Base`](pathname:///api/0.3/Marten/Schema/Field/Base.html) is fairly simply and looks like this: + +```crystal +def initialize( + @id : ::String, + @required : ::Bool = true +) +end +``` + +Depending on your field requirements, you might want to override this method completely in order to support additional parameters (such as default validation-related options for example). + +#### `validate` + +The `#validate` method does nothing by default and can be overridden on a per-field class basis in order to implement custom validation logic. This method takes the schema object being validated and the field value as arguments, which allows you to easily run validation checks and to add [validation errors](../validations) to the schema object. + +For example: + +```crystal +def validate(schema, value) + return if !value.is_a?(::String) + + if !EmailValidator.valid?(value) + schema.errors.add(id, "Provide a valid email address") + end +end +``` + +### An example + +Let's consider the use case of the "email" field highlighted in [Subclassing existing schema fields](#subclassing-existing-schema-fields). The exact same field could be implemented from scratch with the following snippet: + +```crystal +class EmailField < Marten::Schema::Field::Base + getter max_size + getter min_size + + def initialize( + @id : ::String, + @required : ::Bool = true, + @max_size : ::Int32? = 254, + @min_size : ::Int32? = nil, + @strip : ::Bool = true + ) + end + + def deserialize(value) : ::String? + strip? ? value.to_s.strip : value.to_s + end + + def serialize(value) : ::String? + value.try(&.to_s) + end + + def strip? + @strip + end + + def validate(schema, value) + return if !value.is_a?(::String) + + if !min_size.nil? && value.size < min_size.not_nil! + schema.errors.add(id, "The minimum allowed length is #{min_size} characters") + end + + if !max_size.nil? && value.size > max_size.not_nil! + schema.errors.add(id, "The maximum allowed length is #{max_size} characters") + end + + if !EmailValidator.valid?(value) + record.errors.add(id, "Provide a valid email address") + end + end +end +``` diff --git a/docs/versioned_docs/version-0.3/schemas/introduction.md b/docs/versioned_docs/version-0.3/schemas/introduction.md new file mode 100644 index 000000000..441ad94d8 --- /dev/null +++ b/docs/versioned_docs/version-0.3/schemas/introduction.md @@ -0,0 +1,260 @@ +--- +title: Introduction to schemas +description: Learn how to define schemas and use them in handlers. +sidebar_label: Introduction +--- + +Schemas are classes that define how input data should be serialized/deserialized, and validated. Schemas are usually used when processing web requests containing form data or pre-defined payloads. + +## Basic schema definition and usage + +### The schema class + +A schema class describes an _expected_ set of data. It describes the logical structure of this data, what are its expected characteristics, and what are the rules to use in order to identify whether it is valid or not. Schemas classes must inherit from the [`Marten::Schema`](pathname:///api/0.3/Marten/Schema.html) base class and they must define "fields" through the use of a `field` macro. These fields allow to define what data is expected by the schema, and how it is validated. + +For example, the following snippet defines a simple `ArticleSchema` schema: + +```crystal +class ArticleSchema < Marten::Schema + field :title, :string, max_size: 128 + field :content, :string + field :published_at, :date_time, required: false +end +``` + +In the above example, `title`, `content`, and `published_at` are fields of the `ArticleSchema` schema. This schema is very simple, but it already defines a set of validation rules that could be used to validate any data set the schema is applied to: + +* the `title` field is required, it must be a string that do not exceed 128 characters +* the `content` field is required and must be a string as well +* the `published_at` field is a date time that is _not_ required + +### Using schemas + +Schemas can theoretically be used to process any kind of data, including a request's data. This makes them ideal when it comes to processing form inputs data or JSON payloads for example. + +When used as part of [handlers](../handlers-and-http/introduction), and especially when processing HTML forms, schemas will usually be initialized and used to render a form when `GET` requests are submitted to the considered handler. Processing the actual form data will usually be done in the same handler when `POST` requests are submitted. + +For example, the handler in the following snippets displays a schema when a `GET` request is processed, and it validates the incoming data using the schema when the request is a `POST`: + +```crystal +class ArticleCreateHandler < Marten::Handler + @schema : ArticleSchema? + + def get + render("article_create.html", context: { schema: schema }) + end + + def post + if schema.valid? + article = Article.new(schema.validated_data) + article.save! + + redirect(reverse("home")) + else + render("article_create.html", context: { schema: schema }) + end + end + + private def schema + @schema ||= ArticleSchema.new(request.data) + end +end +``` + +Let's break it down a bit more: + +* when the incoming request is a `GET`, the handler will simply render the `article_create.html` template, and initialize the schema (instance of `ArticleSchema`) with any data currently present in the request object (which is returned by the `#request` method). This schema object is made available to the template context +* when the incoming request is a `POST`, it will initialize the schema and try to see if it is valid considering the incoming data (using the [`#valid?`](pathname:///api/0.3/Marten/Core/Validation.html#valid%3F(context%3ANil|String|Symbol%3Dnil)-instance-method) method). If it's valid, then a new `Article` record will be created using the schema's validated data ([`#validated_data`](pathname:///api/0.3/Marten/Schema.html#validated_data%3AHash(String%2CBool|Float64|Int64|JSON%3A%3AAny|JSON%3A%3ASerializable|Marten%3A%3AHTTP%3A%3AUploadedFile|String|Time|Time%3A%3ASpan|UUID|Nil)-instance-method)), and the user will be redirect to a home page. Otherwise, the `article_create.html` template will be rendered again with the invalid schema in the associated context + +It should be noted that templates can easily interact with schema objects in order to introspect them and render a corresponding HTML form. In the above example, the schema could be used as follows to render an equivalent form in the `article_create.html` template: + +```html +
+ + +
+
+ + {% for error in schema.title.errors %}

{{ error.message }}

{% endfor %} +
+ +
+
+ + {% for error in schema.content.errors %}

{{ error.message }}

{% endfor %} +
+ +
+
+ + {% for error in schema.published_at.errors %}

{{ error.message }}

{% endfor %} +
+ +
+ +
+
+``` + +:::tip +Some [generic handlers](../handlers-and-http/generic-handlers) allow to conveniently process schemas in handlers. This is the case for the [`Marten::Handlers::Schema`](../handlers-and-http/reference/generic-handlers#processing-a-schema), the [`Marten::Handlers::RecordCreate`](../handlers-and-http/reference/generic-handlers#creating-a-record), and the [`Marten::Handlers::RecordUpdate`](../handlers-and-http/reference/generic-handlers#updating-a-record) generic handlers for example. +::: + +Note that schemas can be used for other things than processing form data. For example, they can also be used to process JSON payloads as part of API endpoints: + +```crystal +class API::ArticleCreateHandler < Marten::Handler + def post + schema = ArticleCreateHandler.new(request.data) + + if schema.valid? + article = Article.new(schema.validated_data) + article.save! + + created = true + else + created = false + end + + json({created: created}) + end +end +``` + +:::info +The `#data` method of an HTTP request object returns a hash-like object containing the request data: this object is automatically initialized from any form data or JSON data contained in the request body. +::: + +## Schema fields + +Schema classes must define _fields_. Fields allow to specify the expected attributes of a schema and they indicate how to validate incoming data sets. They are defined through the use of the `field` macro. + +For example: + +```crystal +class ArticleSchema < Marten::Schema + field :title, :string, max_size: 128 + field :content, :string + field :published_at, :date_time, required: false +end +``` + +### Field ID and field type + +Pretty much like model fields, every field in a schema class must contain two mandatory positional arguments: a field identifier and a field type. + +The field identifier is used by Marten to determine the name of the corresponding key in any data set objects that should be validated by the schema. + +The field type determines a few other things: + +* the type of the expected value in the validated data set +* how the field is serialized and deserialized +* how field values are actually validated + +Marten provides numerous built-in schema field types that cover common web development needs. The complete list of supported fields is covered in the [schema fields reference](./reference/fields). + +:::note +It is possible to write custom schema fields and to use them in your schema definitions. See [How to create custom schema fields](./how-to/create-custom-schema-fields) for more details regarding this capability. +::: + +### Common field options + +In addition to their identifiers and types, fields can take keyword arguments that allow to further configure their behaviours and how they are validated. These keyword arguments are optional and they are shared across all the available fields. + +#### `required` + +The `required` argument allows to define whether a field is mandatory or not. The default value for this argument is `true`. + +The presence of mandatory fields is automatically enforced by schemas: if a mandatory field is missing in a data set, then a corresponding error will be generated by the schema. + +## Validations + +One of the key characteristics of schemas is that they allow you to validate any incoming data and request parameters. As mentioned previously, the rules that are used to perform this validation can be inherited from the fields in your schema, depending on the options you used (for example fields using `required: true` will make the associated data validation fail if the field value is not present). They can also be explicitly specified in your schema class, which is useful if you need to implement custom validation logics. + +For example: + +```crystal +class SignUpSchema < Marten::Schema + field :email, :string, max_size: 254 + field :password1, :string, max_size: 128, strip: false + field :password2, :string, max_size: 128, strip: false + + validate :validate_password + + def validate_password + return unless validated_data["password1"]? && validated_data["password2"]? + + if validated_data["password1"] != validated_data["password2"] + errors.add("The two password fields do not match") + end + end +end +``` + +Schema validations are always triggered by the use of the [`#valid?`](pathname:///api/0.3/Marten/Core/Validation.html#valid%3F(context%3ANil|String|Symbol%3Dnil)-instance-method) or [`#invalid?`](pathname:///api/0.3/Marten/Core/Validation.html#invalid%3F(context%3ANil|String|Symbol%3Dnil)-instance-method) methods: these methods return `true` or `false` depending on whether the data is valid or invalid. + +Please head over to the [Schema validations](./validations) guide in order to learn more about schema validations and how to customize it. + +## Accessing validated data + +After performing [schema validations](#validations) (ie. after calling [`#valid?`](pathname:///api/0.3/Marten/Core/Validation.html#valid%3F(context%3ANil|String|Symbol%3Dnil)-instance-method) or [`#invalid?`](pathname:///api/0.3/Marten/Core/Validation.html#invalid%3F(context%3ANil|String|Symbol%3Dnil)-instance-method) on a schema object), accessing the validated data is often necessary. For instance, you may need to persist the validated data as part of a model record. To achieve this, you can make use of the [`#validated_data`](pathname:///api/0.3/Marten/Schema.html#validated_data%3AHash(String%2CBool|Float64|Int64|JSON%3A%3AAny|JSON%3A%3ASerializable|Marten%3A%3AHTTP%3A%3AUploadedFile|String|Time|Time%3A%3ASpan|UUID|Nil)-instance-method) method, which is accessible in all schema instances. + +This method provides access to a hash that contains the deserialized and validated field values of the schema. For instance, let's consider the example of the `ArticleSchema` schema [mentioned earlier](#the-schema-class): + +```crystal +schema = ArticleSchema.new(Marten::Schema::DataHash{"title" => "Test article", "content" => "Test content"}) +schema.valid? # => true + +schema.validated_data["title"] # => "Test article" +schema.validated_data["content"] # => "Test content" +``` + +It is important to note that accessing values using [`#validated_data`](pathname:///api/0.3/Marten/Schema.html#validated_data%3AHash(String%2CBool|Float64|Int64|JSON%3A%3AAny|JSON%3A%3ASerializable|Marten%3A%3AHTTP%3A%3AUploadedFile|String|Time|Time%3A%3ASpan|UUID|Nil)-instance-method) as shown in the above example is not type-safe. The [`#validated_data`](pathname:///api/0.3/Marten/Schema.html#validated_data%3AHash(String%2CBool|Float64|Int64|JSON%3A%3AAny|JSON%3A%3ASerializable|Marten%3A%3AHTTP%3A%3AUploadedFile|String|Time|Time%3A%3ASpan|UUID|Nil)-instance-method) hash can return any supported schema field values, and as a result, you may need to utilize the [`#as`](https://crystal-lang.org/reference/syntax_and_semantics/as.html) pseudo-method to handle the fetched validated data appropriately, depending on how and where you intend to use it. + +To palliate this, Marten automatically defines type-safe methods that you can utilize to access your validated schema field values: + +* `#` returns a nillable version of the `` field value +* `#!` returns a non-nillable version of the `` field value +* `#?` returns a boolean indicating if the `` field has a value + +For example: + +```crystal +schema = ArticleSchema.new(Marten::Schema::DataHash{"title" => "Test article"}) +schema.valid? # => true + +schema.title # => "Test article" +schema.title! # => "Test article" +schema.title? # => true + +schema.content # => nil +schema.content! # => raises NilAssertionError +schema.content? # => false +``` + +## Callbacks + +It is possible to define callbacks in your schema in order to bind methods and logics to specific events in the life cycle of your schema objects. Presently, schemas support callbacks related to validation only: `before_validation` and `after_validation` + +`before_validation` callbacks are called before running validation rules for a given schema while `after_validation` callbacks are executed after. They can be used to alter the validated data once the validation is done for example. + +```crystal +class ArticleSchema < Marten::Schema + field :title, :string, max_size: 128 + field :content, :string + field :published_at, :date_time, required: false + + before_validation :run_pre_validation_logic + after_validation :run_post_validation_logic + + private def run_pre_validation_logic + # Do something before the validation + end + + private def run_post_validation_logic + # Do something after the validation + end +end +``` + +The use of methods like [`#valid?`](pathname:///api/0.3/Marten/Core/Validation.html#valid%3F(context%3ANil|String|Symbol%3Dnil)-instance-method) or [`#invalid?`](pathname:///api/0.3/Marten/Core/Validation.html#invalid%3F(context%3ANil|String|Symbol%3Dnil)-instance-method) will trigger validation callbacks. See [Schema validations](./validations) for more details. diff --git a/docs/versioned_docs/version-0.3/schemas/reference/fields.md b/docs/versioned_docs/version-0.3/schemas/reference/fields.md new file mode 100644 index 000000000..12119f9dd --- /dev/null +++ b/docs/versioned_docs/version-0.3/schemas/reference/fields.md @@ -0,0 +1,130 @@ +--- +title: Schema fields +description: Schema fields reference. +--- + +This page provides a reference for all the available field options and field types that can be used when defining schemas. + +## Common field options + +The following field options can be used for all the available field types when declaring schema fields using the `field` macro. + +### `required` + +The `required` argument can be used to specify whether a schema field is required or not required. The default value for this argument is `true`. + +## Field types + +### `bool` + +A `bool` field allows validating boolean values. + +### `date_time` + +A `date_time` field allows validating date time values. Fields using this type are converted to `Time` objects in Crystal. + +### `date` + +A `date` field allows validating date values. Fields using this type are converted to `Time` objects in Crystal. + +### `duration` + +A `duration` field allows validating duration values, which map to [`Time::Span`](https://crystal-lang.org/api/Time/Span.html) objects in Crystal. `duration` fields expect serialized values to be in the `DD.HH:MM:SS.nnnnnnnnn` format (with `n` corresponding to nanoseconds) or in the [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601#Durations) format (eg. `P3DT2H15M20S`, which corresponds to a `3.2:15:20` time span). + +### `email` + +An `email` field allows validating email address values. In addition to the [common field options](#common-field-options), such fields support the following arguments: + +#### `max_size` + +The `max_size` argument allows defining the maximum size allowed for the email address string. The default value for this argument is `254` (in accordance with RFCs 3696 and 5321). + +#### `min_size` + +The `min_size` argument allows defining the minimum size allowed for the email address string. The default value for this argument is `nil`, which means that the minimum size is not validated by default. + +#### `strip` + +The `strip` argument allows defining whether the string value should be stripped of leading and trailing whitespaces. The default is `true`. + +### `file` + +A `file` field allows validating uploaded files. In addition to the [common field options](#common-field-options), such fields support the following arguments: + +#### `allow_empty_files` + +The `allow_empty_files` argument allows defining whether empty files are allowed or not when files are validated. The default value is `false`. + +#### `max_name_size` + +The `max_name_size` argument allows defining the maximum file name size allowed. The default value is `nil`, which means that uploaded file name sizes are not validated. + +### `float` + +A `float` field allows validating float values. Fields using this type are converted to `Float64` objects in Crystal. In addition to the [common field options](#common-field-options), such fields support the following arguments: + +#### `max_value` + +The `max_value` argument allows defining the maximum value allowed. The default value for this argument is `nil`, which means that the maximum value is not validated by default. + +#### `min_value` + +The `min_value` argument allows defining the minimum value allowed. The default value for this argument is `nil`, which means that the minimum value is not validated by default. + +### `int` + +An `int` field allows validating integer values. Fields using this type are converted to `Int64` objects in Crystal. In addition to the [common field options](#common-field-options), such fields support the following arguments: + +#### `max_value` + +The `max_value` argument allows defining the maximum value allowed. The default value for this argument is `nil`, which means that the maximum value is not validated by default. + +#### `min_value` + +The `min_value` argument allows defining the minimum value allowed. The default value for this argument is `nil`, which means that the minimum value is not validated by default. + +### `json` + +A `json` field allows validating JSON values, which are automatically parsed to [`JSON::Any`](https://crystal-lang.org/api/JSON/Any.html) objects. Additionally, it is also possible to leverage the [`serializable`](#serializable) option in order to specify a class that makes use of [`JSON::Serializable`](https://crystal-lang.org/api/JSON/Serializable.html). When doing so, the parsing of the JSON values will result in the initialization of the corresponding serializable objects: + +```crystal +class MySerializable + include JSON::Serializable + + property a : Int32 | Nil + property b : String | Nil +end + +class MySchema < Marten::Schema + # Other fields... + field :metadata, :json, serializable: MySerializable +end + +schema = MySchema.new(Marten::Schema::DataHash{"metadata" => %{{"a": 42, "b": "foo"}}}) +schema.valid? # => true +schema.metadata! # => MySerializable object +``` + +#### `serializable` + +The `serializable` arguments allows to specify that a class making use of [`JSON::Serializable`](https://crystal-lang.org/api/JSON/Serializable.html) should be used in order to parse the JSON values for the schema field at hand. When specifying a `serializable` class, the values returned for the considered schema fields will be instances of that class instead of [`JSON::Any`](https://crystal-lang.org/api/JSON/Any.html) objects. + +### `string` + +A `string` field allows validating string values. In addition to the [common field options](#common-field-options), such fields support the following arguments: + +#### `max_size` + +The `max_size` argument allows defining the maximum size allowed for the string. The default value for this argument is `nil`, which means that the maximum size is not validated by default. + +#### `min_size` + +The `min_size` argument allows defining the minimum size allowed for the string. The default value for this argument is `nil`, which means that the minimum size is not validated by default. + +#### `strip` + +The `strip` argument allows defining whether the string value should be stripped of leading and trailing whitespaces. The default is `true`. + +### `uuid` + +A `uuid` field allows validating Universally Unique IDentifiers (UUID) values. Fields using this type are converted to `UUID` objects in Crystal. diff --git a/docs/versioned_docs/version-0.3/schemas/validations.md b/docs/versioned_docs/version-0.3/schemas/validations.md new file mode 100644 index 000000000..e3493bdb7 --- /dev/null +++ b/docs/versioned_docs/version-0.3/schemas/validations.md @@ -0,0 +1,148 @@ +--- +title: Schema validations +description: Learn how to validate data with schemas. +sidebar_label: Validations +--- + +The main goal of schemas is to validate data and request parameters. As such, schemas provide a convenient mechanism allowing to define validation rules. These rules can be inherited from the fields in your schema depending on the options you used and the type of your fields. They can also be explicitly specified in your schema class, which is useful if you need to implement custom validation logics. + + +## Overview + +### A short example + +Let's consider the following example: + +```crystal +class UserSchema < Marten::Schema + field :name, :string, required: true, max_size: 128 +end +``` + +In the above snippet, a `UserSchema` schema is defined and it is specified that the `name` field must be present (`required: true`) and that the associated value cannot exceed 128 characters. + +Given these characteristics, it is possible to initialize `UserSchema` instances and validate data through the use of the `#valid?` method: + +```crystal +schema_1 = UserSchema.new(Marten::Schema::DataHash.new) +schema_1.valid? # => false + +schema_2 = UserSchema.new(Marten::Schema::DataHash{ "name" => "0" * 200) }) +schema_2.valid? # => false + +schema_3 = UserSchema.new(Marten::Schema::DataHash{ "name" => "John Doe") }) +schema_3.valid? # => true +``` + +As you can see in the above examples, the first two schemas are invalid because either the `name` field is not specified or because its value exceeds the maximum characters limit. The last schema is valid though because it has a `name` field that is less than 128 characters. + +### Running schema validations + +As highlighted in the previous section, schema validation rules will be executed when calling the `#valid?` and `#invalid?` methods: these methods return `true` or `false` depending on whether the data is valid or invalid. + +```crystal +schema = UserSchema.new(Marten::Schema::DataHash.new) +schema.valid? # => false +schema.invalid? # => true +``` + +## Field validation rules + +As mentioned previously, fields can contribute validation rules to your schemas. These validation rules can be inherited: + +* from the field type itself: some fields will validate that values are of a specific type (for example a `uuid` field will not validate values that don't correspond to valid UUIDs) +* from the field options you define (for example fields using `required: true` will result in errors if the field is missing from the validated data) + +Please refer to the [fields reference](./reference/fields) to learn more about the supported field types and their associated options. + +## Custom validation rules + +Custom validation rules can be defined through the use of the `#validate` macro. This macro lets you configure the name of a validation method that should be called when a schema instance is validated. Inside this method, you can implement any validation logic that you might require and add errors to the schema instance if the data is invalid. + +For example: + +```crystal +class SignUpSchema < Marten::Schema + field :email, :string, max_size: 254 + field :password1, :string, max_size: 128, strip: false + field :password2, :string, max_size: 128, strip: false + + validate :validate_password + + def validate_password + return unless validated_data["password1"]? && validated_data["password2"]? + + if validated_data["password1"] != validated_data["password2"] + errors.add("The two password fields do not match") + end + end +end +``` + +In the above snippet, a custom validation method ensures that the `password1` and `password2` fields have the exact same value. If that's not the case, then a specific error (that is not associated with any fields) is added to the schema instance (which makes it invalid). It's interesting to note the use of the `#validated_data` method here: this method returns a hash of all the values that were previously sanitized and validated. You can make use of it when defining custom validation rules: indeed, these rules always run _after_ all the fields have been individually validated first. + +:::important +You can define multiple validation rules in your schema classes. When doing so, don't forget that these custom validation rules are called in the order they are defined. +::: + +## Validation errors + +Methods like `#valid?` or `#invalid?` only let you know whether a schema instance is valid or invalid for a specific data set. But you'll likely want to know exactly what are the actual errors or how to add new ones. + +As such, every schema instance has an associated error set, which is an instance of [`Marten::Core::Validation::ErrorSet`](pathname:///api/0.3/Marten/Core/Validation/ErrorSet.html). + +### Inspecting errors + +A schema instance error set lets you access all the errors of a specific schema instance. For example: + +```crystal +schema = UserSchema.new(Marten::Schema::DataHash.new) + +schema.valid? +# => false + +schema.errors.size +# => 1 + +schema.errors +# => #]> +``` + +As you can see, the error set gives you the ability to know how many errors are affecting your schema instance. Each error provides some additional information as well: + +* the associated field name (which can be `nil` if the error is global) +* the error message +* the error type, which is optional (`required` in the previous example) + +You can also access the errors that are associated with a specific field very easily by using the `#[]` method: + +```crystal +schema.errors[:name] +# => [#] +``` + +Global errors (errors affecting the whole schema instances or multiple fields at once) can be listed through the use of the `#global` method. + +### Adding errors + +Errors can be added to an error set through the use of the `#add` method. This method takes a field name, a message, and an optional error type: + +```crystal +schema.errors.add(:name, "Name is invalid") # error type is "invalid" +schema.errors.add(:name, "Name is invalid", type: :invalid_name) # error type is "invalid_name" +``` + +Global errors can be specified through the use of an alternative `#add` method that doesn't take a field name: + +```crystal +schema.errors.add("User is invalid") # error type is "invalid" +schema.errors.add("User is invalid", type: :invalid_user) # error type is "invalid_user" +``` diff --git a/docs/versioned_docs/version-0.3/security.mdx b/docs/versioned_docs/version-0.3/security.mdx new file mode 100644 index 000000000..d6dd7ef64 --- /dev/null +++ b/docs/versioned_docs/version-0.3/security.mdx @@ -0,0 +1,24 @@ +--- +title: Security +--- + +import DocCard from '@theme/DocCard'; + +Security is one of the most important topics to consider when developing web applications. This section covers the security measures that are provided by Marten and how to leverage them in your applications. + +## Guides + +
+
+ +
+
+ +
+
+ +
+
+ +
+
diff --git a/docs/versioned_docs/version-0.3/security/clickjacking.md b/docs/versioned_docs/version-0.3/security/clickjacking.md new file mode 100644 index 000000000..aa89e847d --- /dev/null +++ b/docs/versioned_docs/version-0.3/security/clickjacking.md @@ -0,0 +1,37 @@ +--- +title: Clickjacking protection +description: Learn about clickjacking and how Marten helps protect against this type of attacks. +--- + +This document describes Marten's clickjacking protection mechanism as well as the various tools that you can use in order to configure and make use of it. + +## Overview + +Clickjacking attacks involve a malicious website embedding another unprotected website in a frame. This can lead to users performing unintended actions on the targeted website. + +The best way to mitigate this risk is to rely on the X-Frame-Options header: this header indicates whether or not the protected resource is allowed to be embedded into a frame, and if so under which conditions. The X-Frame-Options header can be set to `DENY` or `SAMEORIGIN`: + +* `DENY` means that the response cannot be displayed inside a frame at all +* `SAMEORIGINS` means that the browser will allow the response to be displayed inside a frame if the site defining the frame is the same as the one serving the actual resource + +## Basic usage + +Marten's clickjacking protection involves using a dedicated middleware: the [X-Frame-Options middleware](../handlers-and-http/reference/middlewares#x-frame-options-middleware). This middleware is automatically added to the [`middleware`](../development/reference/settings#middleware) setting when generating projects via the [`new`](../development/reference/management-commands#new) management command. + +The [X-Frame-Options middleware](../handlers-and-http/reference/middlewares#x-frame-options-middleware) simply sets the X-Frame-Options header in order to prevent the considered Marten website from being inserted into a frame. The value that is used for the X-Frame-Options header depends on the value of the [`x_frame_options`](../development/reference/settings#x_frame_options) setting (whose default value is `DENY`). + +It should be noted that you can decide to disable or enable the use of the [X-Frame-Options middleware](../handlers-and-http/reference/middlewares#x-frame-options-middleware) on a per-handler basis. To do so, you can simply make use of the [`#exempt_from_x_frame_options`](pathname:///api/0.3/Marten/Handlers/XFrameOptions/ClassMethods.html#exempt_from_x_frame_options(exempt%3ABool)%3ANil-instance-method) class method, which takes a single boolean as arguments: + +```crystal +class ProtectedHandler < Marten::Handler + exempt_from_x_frame_options false + + # [...] +end + +class UnprotectedHandler < Marten::Handler + exempt_from_x_frame_options true + + # [...] +end +``` diff --git a/docs/versioned_docs/version-0.3/security/content-security-policy.md b/docs/versioned_docs/version-0.3/security/content-security-policy.md new file mode 100644 index 000000000..5028f2645 --- /dev/null +++ b/docs/versioned_docs/version-0.3/security/content-security-policy.md @@ -0,0 +1,94 @@ +--- +title: Content Security Policy +description: Learn how to configure the Content-Security-Policy (CSP) header. +--- + +Marten offers a convenient mechanism to define the Content-Security-Policy header, which serves as a safeguard against vulnerabilities such as cross-site scripting (XSS) and injection attacks. This mechanism enables the specification of a trusted resource allowlist, enhancing security measures. + +## Overview + +The [Content-Security-Policy](https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP) (CSP) header is a collection of guidelines that the browser follows to allow specific sources for scripts, styles, embedded content, and more. It ensures that only these approved sources are allowed while blocking all other sources. + +Utilizing the Content-Security-Policy header in a web application is a great way to mitigate or eliminate cross-site scripting (XSS) vulnerabilities. By implementing an effective Content-Security-Policy, the inclusion of inline scripts is prevented, and only scripts from trusted sources in separate files are allowed. + +## Basic usage + +Marten's Content-Security-Policy mechanism involves using a dedicated middleware: the [Content-Security-Policy middleware](../handlers-and-http/reference/middlewares#content-security-policy-middleware). To ensure that your project is using this middleware, you can add the [`Marten::Middleware::ContentSecurityPolicy`](pathname:///api/0.3/Marten/Middleware/ContentSecurityPolicy.html) class to the [`middleware`](../development/reference/settings#middleware) setting as follows: + +```crystal title="config/settings/base.cr" +Marten.configure do |config| + config.middleware = [ + // highlight-next-line + Marten::Middleware::ContentSecurityPolicy, + # Other middlewares... + Marten::Middleware::Session, + Marten::Middleware::Flash, + Marten::Middleware::I18n, + ] +end +``` + +The [Content-Security-Policy middleware](../handlers-and-http/reference/middlewares#content-security-policy-middleware) guarantees the presence of the Content-Security-Policy header in the response's headers. By default, the middleware will include a Content-Security-Policy header that corresponds to the policy defined in the [`content_security_policy`](../development/reference/settings#content-security-policy-settings) settings. However, if a [`Marten::HTTP::ContentSecurityPolicy`](pathname:///api/0.3/Marten/HTTP/ContentSecurityPolicy.html) object is explicitly assigned to the request object, it will take precedence over the default policy and be used instead. + +When enabling the [Content-Security-Policy middleware](../handlers-and-http/reference/middlewares#content-security-policy-middleware), it is recommended to define a default Content-Security-Policy by leveraging the [`content_security_policy`](../development/reference/settings#content-security-policy-settings) settings. For example: + +```crystal title="config/settings/base.cr" +Marten.configure do |config| + config.content_security_policy.default_policy.default_src = [:self, "example.com"] + config.content_security_policy.default_policy.script_src = [:self, :https] +end +``` + +## Disabling the CSP header in specific handlers + +You can decide to disable or enable the use of the Content-Security-Policy header on a per-[handler](../handlers-and-http) basis. To do so, you can simply make use of the [`#exempt_from_content_security_policy`](pathname:///api/0.3/Marten/Handlers/ContentSecurityPolicy/ClassMethods.html#exempt_from_content_security_policy(exempt:Bool):Nil-instance-method) class method, which takes a single boolean as argument: + +```crystal +class ProtectedHandler < Marten::Handler + exempt_from_content_security_policy false + + # [...] +end + +class UnprotectedHandler < Marten::Handler + exempt_from_content_security_policy true + + # [...] +end +``` + +## Overriding the CSP header in specific handlers + +Sometimes you may also need to override the content of the Content-Security-Policy header on a per-[handler](../handlers-and-http) basis. To do so, you can make use of the [`#content_security_policy`](pathname:///api/0.3/Marten/Handlers/ContentSecurityPolicy/ClassMethods.html#content_security_policy(%26content_security_policy_block%3AHTTP%3A%3AContentSecurityPolicy->)-instance-method) class method, which yields a [`Marten::HTTP::ContentSecurityPolicy`](pathname:///api/0.3/Marten/HTTP/ContentSecurityPolicy.html) object that you can configure (by adding/modifying/removing CSP directives) for the handler at hand. For example: + +```crystal +class ProtectedHandler < Marten::Handler + content_security_policy do |csp| + csp.default_src = {:self, "example.com"} + end + + # [...] +end +``` + +## Using a CSP nonce + +CSP nonces serve as a valuable tool to enable the execution or rendering of specific elements, such as inline script or style tags, by the browser. When a tag contains the correct nonce value in a `nonce` attribute, the browser grants permission for its execution or rendering, while blocking others that lack the expected nonce value. + +You can configure Marten so that it automatically adds a nonce to an explicit set of Content-Security-Policy directives. This can be achieved by specifying the list of intended CSP directives in the [`content_security_policy.nonce_directives`](../development/reference/settings#nonce_directives) setting. For example: + +```crystal title="config/settings/base.cr" +Marten.configure do |config| + config.content_security_policy.nonce_directives = ["script-src", "style-src"] +end +``` + +For example, if this setting is set to `["script-src", "style-src"]`, a `nonce-` value will be added to the `script-src` and `style-src` directives in the Content-Security-Policy header value. The nonce is a randomly generated Base64 value (generated through the use of [`Random::Secure#urlsafe_base64`](https://crystal-lang.org/api/Random.html#urlsafe_base64(n:Int=16,padding=false):String-instance-method)). + +To make the browser do anything with the nonce value, you will need to include it in the attributes of the tags that you wish to mark as safe. In this light, you can use the [`Marten::HTTP::Request#content_security_policy_nonce`](pathname:///api/0.3/Marten/HTTP/Request.html#content_security_policy_nonce-instance-method) method, which returns the CSP nonce value for the current request. This method can also be called from within [templates](../templates), making it easy to generate `script` or `style` tags containing the right `nonce` attribute: + +```html + +``` diff --git a/docs/versioned_docs/version-0.3/security/csrf.md b/docs/versioned_docs/version-0.3/security/csrf.md new file mode 100644 index 000000000..dc4ebb31f --- /dev/null +++ b/docs/versioned_docs/version-0.3/security/csrf.md @@ -0,0 +1,116 @@ +--- +title: Cross-Site Request Forgery protection +description: Learn about Cross-Site Request Forgeries (CSRF) attacks and how to protect your application from them. +sidebar_label: CSRF protection +--- + +This document describes Marten's Cross-Site Request Forgery (CSRF) protection mechanism as well as the various tools that you can use in order to configure and make use of it. + +## Overview + +Cross-Site Request Forgery (CSRF) attacks generally involve a malicious website trying to perform actions on a web application on behalf of an already authenticated user. Marten provides a built-in mechanism in order to protect your applications from this kind of attack. This mechanism is useful for protecting endpoints that handle "unsafe" HTTP requests (ie. requests whose methods are not `GET`, `HEAD`, `OPTIONS`, or `TRACE`). + +:::caution +The CSRF protection ignores safe HTTP requests. As such, you should ensure that those are side effect free. +::: + +The CSRF protection provided by Marten is based on the verification of a token that must be provided for each unsafe HTTP request. This token is stored in the client: Marten sends a token cookie with every HTTP response when the token value is requested in handlers ([`#get_csrf_token`](pathname:///api/0.3/Marten/Handlers/RequestForgeryProtection.html#get_csrf_token-instance-method) method) or templates (eg. through the use of the [`csrf_token`](../templates/reference/tags#csrf_token) tag). It should be noted that the actual value of the token cookie changes every time an HTTP response is returned to the client: this is because the actual secret token is scrambled using a mask that changes for every request where the CSRF token is requested and used. + +The token value must be specified when submitting unsafe HTTP requests: this can be done either in the data itself (by specifying a `csrftoken` input) or by using a specific header (X-CSRF-Token). When receiving this value, Marten compares it to the token cookie value: if the tokens are not valid, or if there is a mismatch, then this means that the request is malicious and that it must be rejected (which will result in a 403 error). + +Finally, it should be noted that a few additional checks can be performed in addition to the token verification: + +* in order to protect against cross-subdomain attacks, the HTTP request host will be verified in order to ensure that it is either part of the allowed hosts ([`allowed_hosts`](../development/reference/settings#allowed_hosts) setting) or that the value of the Origin header matches the configured trusted origins ([`csrf.trusted_origins`](../development/reference/settings#trusted_origins) setting) +* the Referer header will also be checked for HTTPS requests (if the Origin header is not set) in order to prevent subdomains to perform unsafe HTTP requests on the protected web applications (unless those subdomains are explicitly allowed as part of the [`csrf.trusted_origins`](../development/reference/settings#trusted_origins) setting) + +The Cross-Site Request Forgery protection provided by Marten happens at the handler level automatically. This protection is implemented in the [`Marten::Handlers::RequestForgeryProtection`](pathname:///api/0.3/Marten/Handlers/RequestForgeryProtection.html) module. + +## Basic usage + +You should first ensure that the CSRF protection is enabled, which is the case by default when projects are generated through the use of the [`new`](../development/reference/management-commands#new) management command. That being said, if the CSRF protection is globally disabled (when the [`csrf.protection_enabled`](../development/reference/settings#protection_enabled) setting is set to `false`) you need to ensure that your handler enables it _locally_. For example: + +```crystal +class MyHandler < Marten::Handler + protect_from_forgery true + + # [...] +end +``` + +Then all you need to do is to ensure that you include the CSRF token when submitting unsafe HTTP requests to your web application. How to do that depends on _how_ you intend to submit these requests. + +### Using CSRF protection with forms + +If you need to embed the CSRF token into a form that is generated by a [template](../templates), then you can make use of the [`csrf_token`](../templates/reference/tags#csrf_token) template tag in order to define a hidden `csrftoken` input. + +For example: + +```html +
+ + + + +
+ +
+
+``` + +:::caution +You should never define a hidden `csrftoken` input in a form that does not target your application directly. This is to prevent your CSRF token from being leaked. +::: + +### Using the CSRF protection with AJAX + +If you need to submit unsafe HTTP requests on the client side using AJAX, then you also need to ensure that the CSRF token is specified in the request. In this light, you can generate requests that include a X-CSRF-Token header with the token value. But you first need to retrieve the CSRF token. To get it you can either: + +* retrieve the CSRF token from the cookies (which can be done only if the [`csrf.cookie_http_only`](../development/reference/settings#cookie_http_only) setting is set to `false`) +* or insert the CSRF token somewhere in your HTML markup (which is the way to go if the [`csrf.cookie_http_only`](../development/reference/settings#cookie_http_only) setting is set to `true`) + +Retrieving the CSRF token from the cookies on the client side can be easily done by using a dedicated library such as the [JavaScript Cookie](https://www.npmjs.com/package/cookie) one: + +```javascript +const csrfToken = Cookies.get("csrftoken"); +``` + +If you can't leverage this technique because the [`csrf.cookie_http_only`](../development/reference/settings#cookie_http_only) setting is set to `true`, then you can also define the CSRF token as a JavaScript variable on the template side (by using the [`csrf_token`](../templates/reference/tags#csrf_token) template tag): + +```html + +``` + +An alternative approach could also involve defining an invisible tag with a data attribute, and retrieving this value in order to define a JavaScript variable containing the token value: + +```html +
+ +``` + +Once you have the CSRF token value, all you need to do is to ensure that a X-CSRF-Token header is set with this value in all the unsafe HTTP requests you are issuing. + +## Configuring the CSRF protection + +The CSRF protection is enabled by default and can be configured through the use of a [dedicated set of settings](../development/reference/settings#csrf-settings). These settings can be used to enable or disable the protection globally, tweak some of the parameters of the CSRF token cookie, change the trusted origins, etc. + +## Enabling or disabling the protection on a per-handler basis + +Regardless of the value of the [`csrf.protection_enabled`](../development/reference/settings#protection_enabled) setting, it is possible to enable or disable the CSRF protection on a per-handler basis. This can be achieved through the use of the [`#protect_from_forgery`](pathname:///api/0.3/Marten/Handlers/RequestForgeryProtection/ClassMethods.html#protect_from_forgery(protect%3ABool)%3ANil-instance-method) class method, which takes a single boolean as arguments: + +```crystal +class ProtectedHandler < Marten::Handler + protect_from_forgery true + + # [...] +end + +class UnprotectedHandler < Marten::Handler + protect_from_forgery false + + # [...] +end +``` diff --git a/docs/versioned_docs/version-0.3/security/introduction.md b/docs/versioned_docs/version-0.3/security/introduction.md new file mode 100644 index 000000000..efbac7904 --- /dev/null +++ b/docs/versioned_docs/version-0.3/security/introduction.md @@ -0,0 +1,57 @@ +--- +title: Security in Marten +description: Learn about the main security features provided by the Marten framework. +sidebar_label: Introduction +--- + +This document describees the main security features that are provided by the Marten web framework. + +## Cross-Site Request Forgery protection + +Cross-Site Request Forgery (CSRF) attacks generally involve a malicious website trying to perform actions on a web application on behalf of an already authenticated user. + +Marten comes with a built-in [CSRF protection mechanism](./csrf) that is automatically enabled for your [handlers](../handlers-and-http). The use of this CSRF protection mechanism is controlled by a set of [dedicated settings](../development/reference/settings#csrf-settings). + +:::caution +You should be careful when tweaking those settings and avoid disabling this protection unless this is absolutely necessary. +::: + +The CSRF protection provided by Marten is based on the verification of a token that must be provided for each unsafe HTTP request (ie. requests whose methods are not `GET`, `HEAD`, `OPTIONS`, or `TRACE`). This token is stored in the client cookies and it must be specified when submitting unsafe requests (either in the data itself or using a specific header): if the tokens are not valid, or if the cookie-based token does not match the one provided in the data, then this means that the request is malicious and that it must be rejected. + +You can learn about the CSRF protection provided by Marten and the associated tools in the [dedicated documentation](./csrf). + +## Clickjacking protection + +Clickjacking attacks involve a malicious website embedding another unprotected website in a frame. This can lead to users performing unintended actions on the targeted website. + +Marten comes with a built-in [clickjacking protection mechanism](./clickjacking), that involves using a dedicated middleware (the [X-Frame-Options middleware](../handlers-and-http/reference/middlewares#x-frame-options-middleware)). This middleware is always automatically enabled for projects that are generated via the [`new`](../development/reference/management-commands#new) management command and, as its name implies, it involves setting the X-Frame-Options header in order to prevent the considered Marten website from being inserted into a frame. + +You can learn about the clickjacking protection provided by Marten and the associated tools in the [dedicated documentation](./clickjacking). + +## Cross Site Scripting protection + +Cross Site Scripting (XSS) attacks involve a malicious user injecting client-side scripts into the browser of another user. This usually happens when rendering database-stored HTML data or when generating HTML contents and displaying it in a browser: if these HTML contents are not properly sanitized, then this can allow an attacker's JavaScript to be executed in the browser. + +To prevent this, Marten [templates](../templates) automatically escape HTML contents in variable outputs, unless those are marked as "safe". You can learn more about this capability in [Auto-Escaping](../templates/introduction#auto-escaping). + +It should be noted that this auto-escaping mechanism can be disabled using a specific [filter](../templates/reference/filters#safe) if needed, but you should be aware of the risks while doing so and ensure that your HTML contents are properly sanitized in order to avoid XSS vulnerabilities. + +## HTTP Host Header attacks protection + +HTTP Host Header attacks happen when websites that handle the value of the Host header (eg. in order to generate fully qualified URLs) trust this header value implicitly and don't verify it. + +Marten implements a protection mechanism against this type of attack by validating the Host header against a set of explicitly allowed hosts that must be specified in the [`allowed_hosts`](../development/reference/settings#allowed_hosts) setting. The X-Forwarded-Host header can also be used to determine the host if the use of this header is enabled ([`use_x_forwarded_host`](../development/reference/settings#use_x_forwarded_host) setting). + +## SQL injection protection + +SQL injection attacks happen when a malicious user is able to execute arbitrary SQL queries on a database, which usually occurs when submitting input data to a web application. This can lead to database records being leaked and/or altered. + +The [query sets](../models-and-databases/queries) API provided by Marten generates SQL code by using query parameterization. This means that the actual code of a query is defined separately from its parameters, which ensures that any user-provided parameter is escaped by the considered database driver before the query is executed. + +## Content Security Policy + +The [Content-Security-Policy](https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP) (CSP) header is a collection of guidelines that the browser follows to allow specific sources for scripts, styles, embedded content, and more. It ensures that only these approved sources are allowed while blocking all other sources. + +Marten comes with a built-in [Content Security Policy mechanism](./content-security-policy), that involves using a dedicated middleware (the [Content-Security-Policy middleware](../handlers-and-http/reference/middlewares#content-security-policy-middleware)). This middleware guarantees the presence of the Content-Security-Policy header in the response's headers. + +You can learn about the Content-Security-Policy header and how to configure it in the [dedicated documentation](./content-security-policy). diff --git a/docs/versioned_docs/version-0.3/static/img/getting-started/tutorial/marten_welcome_page.png b/docs/versioned_docs/version-0.3/static/img/getting-started/tutorial/marten_welcome_page.png new file mode 100644 index 000000000..28ab12ed5 Binary files /dev/null and b/docs/versioned_docs/version-0.3/static/img/getting-started/tutorial/marten_welcome_page.png differ diff --git a/docs/versioned_docs/version-0.3/templates.mdx b/docs/versioned_docs/version-0.3/templates.mdx new file mode 100644 index 000000000..777778e82 --- /dev/null +++ b/docs/versioned_docs/version-0.3/templates.mdx @@ -0,0 +1,43 @@ +--- +title: Templates +--- + +import DocCard from '@theme/DocCard'; + +Templates provide a convenient way to write HTML content that is rendered dynamically. This rendering can involve model records or any other variables you define. + +## Guides + +
+
+ +
+
+ +## How-to's + +
+
+ +
+
+ +
+
+ +
+
+ +## Reference + +
+
+ +
+
+ +
+
+ +
+
diff --git a/docs/versioned_docs/version-0.3/templates/how-to/create-custom-context-producers.md b/docs/versioned_docs/version-0.3/templates/how-to/create-custom-context-producers.md new file mode 100644 index 000000000..03ecccdca --- /dev/null +++ b/docs/versioned_docs/version-0.3/templates/how-to/create-custom-context-producers.md @@ -0,0 +1,28 @@ +--- +title: Create custom context producers +sidebar_label: Create context producers +description: How to create custom context producers. +--- + +Marten has built-in support for common [context producers](../reference/context-producers), but the framework also allows you to write your own context producers that you can leverage as part of your project's templates. This allows you to easily reuse common context values over multiple templates. + +## Defining a context producer + +Defining a context producer involves creating a subclass of the [`Marten::Template::ContextProducer`](pathname:///api/0.3/Marten/Template/ContextProducer.html) abstract class. This abstract class requires that subclasses implement a single [`#produce`](pathname:///api/0.3/Marten/Template/ContextProducer.html#produce(request%3AHTTP%3A%3ARequest%3F%3Dnil)-instance-method) method: this method takes an optional request object as argument and must return either: + +* a hash or a named tuple containing the values to contribute to the template context +* or `nil` if no values can be generated for the passed request + +For example, the following context producer would expose the value of the [`debug`](../../development/reference/settings#debug) setting to all the template contexts being created: + +```crystal +class Debug < Marten::Template::ContextProducer + def produce(request : Marten::HTTP::Request? = nil) + {"debug" => Marten.settings.debug} + end +end +``` + +## Activating context producers + +As mentioned in [Using context producers](../introduction#using-context-producers), context producers classes must be added to the [`templates.context_producers`](../../development/reference/settings#contextproducers) setting in order to be used by the Marten templates engine when initializing new context objects. diff --git a/docs/versioned_docs/version-0.3/templates/how-to/create-custom-filters.md b/docs/versioned_docs/version-0.3/templates/how-to/create-custom-filters.md new file mode 100644 index 000000000..b4f0f9e64 --- /dev/null +++ b/docs/versioned_docs/version-0.3/templates/how-to/create-custom-filters.md @@ -0,0 +1,96 @@ +--- +title: Create custom template filters +sidebar_label: Create custom filters +description: How to create custom template filters. +--- + +Marten has built-in support for common [template filters](../reference/filters), but the framework also allows you to write your own template filters that you can leverage as part of your project's templates. + +## Defining a template filter + +Filters are subclasses of the [`Marten::Template::Filter::Base`](pathname:///api/0.3/Marten/Template/Filter/Base.html) abstract class. They must implement a single `#apply` method: this method takes the value the filter should be applied to (a [`Marten::Template::Value`](pathname:///api/0.3/Marten/Template/Value.html) object wrapping _any_ of the object types supported by templates) and an optional argument that was specified to the filter. + +For example, in the expression `{{ var|test:42 }}`, the `test` filter would be called with the value of the `var` variable and the filter argument `42`. + +Let's say we want to write an `underscore` template filter: this template won't need any arguments and will simply return the "underscore" version of the string representation of the incoming value. Such filter could be defined as follows: + +```crystal +class UnderscoreFilter < Marten::Template::Filter::Base + def apply(value : Marten::Template::Value, arg : Marten::Template::Value? = nil) : Marten::Template::Value + Marten::Template::Value.from(value.to_s.underscore) + end +end +``` + +As you can see, the `#apply` method must return a [`Marten::Template::Value`](pathname:///api/0.3/Marten/Template/Value.html) object. + +Now let's try to write a `chomp` template filter that actually makes use of the specified argument. In this case, the argument will be used to define the suffix that should be removed from the end of the string representation of the incoming value: + +```crystal +class ChompFilter < Marten::Template::Filter::Base + def apply(value : Marten::Template::Value, arg : Marten::Template::Value? = nil) : Marten::Template::Value + raise Marten::Template::Errors::InvalidSyntax.new("The 'chomp' filter requires one argument") if arg.nil? + Marten::Template::Value.from(value.to_s.chomp(arg.not_nil!.to_s)) + end +end +``` + +:::info +You should feel free to raise [`Marten::Template::Errors::InvalidSyntax`](pathname:///api/0.3/Marten/Template/Errors/InvalidSyntax.html) from a filter's `#apply` method: this is especially relevant if the input has an unexpected type or if an argument is missing. That being said, it should be noted that any exception raised from a template filter won't be handled by the template engine and will result in a server error (unless explicitly handled by the application itself). +::: + +### `Marten::Template::Value` objects + +As highlighted previously, template filters mainly interact with [`Marten::Template::Value`](pathname:///api/0.3/Marten/Template/Value.html) objects: they take such objects as parameters (for the incoming value the filter should be applied to and for the optional filter parameter), and they must return such objects as well. + +[`Marten::Template::Value`](pathname:///api/0.3/Marten/Template/Value.html) objects can be created from any supported object by using the `#from` method as follows: + +```crystal +Marten::Template::Value.from("hello") +Marten::Template::Value.from(42) +Marten::Template::Value.from(true) +``` + +These objects are essentially "wrappers" around a real value that is manipulated as part of a template's runtime, and they provide a common interface allowing to interact with these during the template rendering. Your filter implementation can perform checks on the incoming [`Marten::Template::Value`](pathname:///api/0.3/Marten/Template/Value.html) objects if necessary: eg. in order to verify that the underlying value is of the expected type. In this light, it is possible to make use of the `#raw` method to retrieve the real value that is wrapped by the [`Marten::Template::Value`](pathname:///api/0.3/Marten/Template/Value.html) object: + +```crystal +value = Marten::Template::Value.from("hello") +value.raw # => "hello" +``` + +### Filters and HTML auto-escaping + +When writing filters that are intended to operate on strings, it is important to remember that [HTML is automatically escaped](../introduction#auto-escaping) in templates. As such, some string values might be flagged as "safe" and some others as "unsafe": + +* regular `String` values are always assumed to be "unsafe" and will be automatically escaped by Marten's template engine +* safe strings are wrapped in `Marten::Template::SafeString` objects + +This means that if your filter needs to have a different behavior based on the fact that a string is safe or not, then you will have to verify what is the type of the underlying value (by relying on the `#raw` method as explained in the previous section). It is the filter's responsibility to ensure that an incoming "safe string" is returned as a "safe string" as well or simply converted to a regular string that will be auto-escaped. + +Creating safe strings is simply a matter of initializing `Marten::Template::SafeString` from a regular string: + +```crystal +class SafeFilter < Marten::Template::Filter::Base + def apply(value : Marten::Template::Value, arg : Marten::Template::Value? = nil) : Marten::Template::Value + Marten::Template::Value.from(Marten::Template::SafeString.new(value.to_s)) + end +end +``` + +## Registering template filters + +To be able to use custom template filters, you must register them to Marten's global template filters registry. + +To do so, you will have to call the [`Marten::Template::Filter#register`](pathname:///api/0.3/Marten/Template/Filter.html#register(filter_name%3AString|Symbol%2Cfilter_klass%3ABase.class)-class-method) method with the name of the filter you wish to use in templates, and the filter class. + +For example: + +```crystal +Marten::Template::Filter.register("underscore", UnderscoreFilter) +``` + +With the above registration, you could technically use this filter as follows: + +```html +{{ my_var|underscore }} +``` diff --git a/docs/versioned_docs/version-0.3/templates/how-to/create-custom-tags.md b/docs/versioned_docs/version-0.3/templates/how-to/create-custom-tags.md new file mode 100644 index 000000000..e3f8a4ca4 --- /dev/null +++ b/docs/versioned_docs/version-0.3/templates/how-to/create-custom-tags.md @@ -0,0 +1,187 @@ +--- +title: Create custom template tags +sidebar_label: Create custom tags +description: How to create custom template tags. +--- + +Marten has built-in support for common [template tags](../reference/tags), but the framework also allows you to write your own template tags that you can leverage as part of your project's templates. + +## Defining a template tag + +Template tags are subclasses of the [`Marten::Template::Tag::Base`](pathname:///api/0.3/Marten/Template/Tag/Base.html) abstract class. When writing custom template tags, you will usually want to define two methods in your tag classes: the `#initialize` and the `#render` methods. These two methods are called at different moments in a template's lifecycle: + +* the `#initialize` method is used to initialize a template tag object and it is called at **parsing time**: this means that it is the responsibility of this method to ensure that the content of the template tag is valid from a parsing standpoint +* the `#render` method is called at **rendering time** to apply the tag's logic: this means that the method is only called for valid template tag statements that were parsed without errors + +Since tags are created and processed when the template is parsed, they can theoretically be used to implement any kind of behavior. That being said, there are a few patterns that are frequently used when writing tags that you might want to consider to help you get started: + +* **simple tags:** tags outputting a value that can (optionally) be assigned to a new variable +* **inclusion tags:** tags including and rendering other templates +* **closable tags:** tags involving closing statements and doing something with the output of a block + +### Simple tags + +Simple tags usually output a value while allowing this value to be assigned to a new variable (that will be added to the template context). They can eventually take arguments in order to return the right result at rendering time. + +Let's take the example of a `local_time` template tag that outputs the string representation of the local time and that takes one mandatory argument (the [format](https://crystal-lang.org/api/Time/Format.html) used to output the time). Such a template tag could be implemented as follows: + +```crystal +class LocalTimeTag < Marten::Template::Tag::Base + include Marten::Template::Tag::CanSplitSmartly + + @assigned_to : String? = nil + + def initialize(parser : Marten::Template::Parser, source : String) + parts = split_smartly(source) + + if parts.size < 2 + raise Marten::Template::Errors::InvalidSyntax.new( + "Malformed local_time tag: one argument must be provided" + ) + end + + @pattern_expression = Marten::Template::FilterExpression.new(parts[1]) + + # Identify possible assigned variable name. + if parts.size > 2 && parts[-2] == "as" + @assigned_to = parts[-1] + elsif parts.size > 2 + raise Marten::Template::Errors::InvalidSyntax.new( + "Malformed local_time tag: only one argument must be provided" + ) + end + end + + def render(context : Marten::Template::Context) : String + time_pattern = @pattern_expression.resolve(context).to_s + + local_time = Time.local(Marten.settings.time_zone).to_s(time_pattern) + + if @assigned_to.nil? + local_time + else + context[@assigned_to.not_nil!] = local_time + "" + end + end +end +``` + +As you can see template tags are initialized from a parser (instance of [Marten::Template::Parser](pathname:///api/0.3/Marten/Template/Parser.html)) and the raw "source" of the template tag (that is the content between the `{%` and `%}` tag delimiters). The `#initialize` method is responsible for extracting any information that might be necessary to implement the template tag's logic. In the case of the `local_time` template tag, we must take care of a few things: + +* ensure that we have a format specified as argument (and raise an invalid syntax error otherwise) +* initialize a filter expression (instance of [Marten::Template::FilterExpression](pathname:///api/0.3/Marten/Template/FilterExpression.html)) from the format argument: this is necessary because the argument can be a string literal or variable with filters applied to it +* verify if the output of the template tag is assigned to a variable by looking for an `as` statement: if that's the case the name of the variable is persisted in a dedicated instance variable + +The `#render` method is called at rendering time: it takes the current context object as argument and must return a string. In the above example, this method "resolves" the time format expression that was identified at initialization time from the context (which is necessary if it was a variable) and generates the right time representation. If the tag wasn't specified with an `as` variable, then this value is simply returned, otherwise, it is persisted in the context and an empty string is returned. + +### Inclusion tags + +Inclusion tags are similar to simple tags: they can take arguments (mandatory or not), and assign their outputs to variables, but the difference is that they render a template in order to produce the final output. + +Let's take the example of a `list` template tag that outputs the elements of an array in a regular `ul` HTML tag. The template being rendered by such template tag could look like this: + +```html title=path/to/list_tag.html +
    + {% for item in list %} +
  • {{ item }}
  • + {% endfor %} +
+``` + +And the template tag itself could be implemented as follows: + +```crystal +class ListTag < Marten::Template::Tag::Base + include Marten::Template::Tag::CanSplitSmartly + + @assigned_to : String? = nil + + def initialize(parser : Marten::Template::Parser, source : String) + parts = split_smartly(source) + + if parts.size < 2 + raise Marten::Template::Errors::InvalidSyntax.new( + "Malformed list tag: one argument must be provided" + ) + end + + @list_expression = Marten::Template::FilterExpression.new(parts[1]) + + # Identify possible assigned variable name. + if parts.size > 2 && parts[-2] == "as" + @assigned_to = parts[-1] + elsif parts.size > 2 + raise Marten::Template::Errors::InvalidSyntax.new( + "Malformed list tag: only one argument must be provided" + ) + end + end + + def render(context : Marten::Template::Context) : String + template = Marten.templates.get_template("path/to/list_tag.html") + + rendered = "" + + context.stack do |include_context| + include_context["list"] = @list_expression.resolve(context) + rendered = template.render(include_context) + end + + if @assigned_to.nil? + Marten::Template::SafeString.new(rendered) + else + context[@assigned_to.not_nil!] = rendered + "" + end + end +end +``` + +As you can see, the implementation of this tag looks quite similar to the one highlighted in [Simple tags](#simple-tags). The only differences that are worth noting here are: + +1. the argument of the template tag corresponds to the list of items that should be rendered +2. the `#render` method explicitly renders the template mentioned previously by using a context with the "list" object in it (the [`#stack`](pathname:///api/0.3/Marten/Template/Context.html#stack(%26)%3ANil-instance-method) method allows to create a new context where new values are stacked over the existing ones). The output of this rendering operation is either assigned to a variable or returned directly depending on whether the `as` statement was used + +### Closable tags + +Closable tags involve a closing statement, like this is the case for the `{% block %}...{% endblock %}` template tag for example. Usually, such tags will "capture" all the nodes between the opening tag and the closing tag, render them at rendering time, and do something with the output of this rendering. + +To illustrate this, let's take the example of a `spaceless` tag that will remove whitespaces, tabs and new lines between HTML tags. Such a template tag could be implemented as follows: + +```crystal +class SpacelessTag < Marten::Template::Base + @inner_nodes : Marten::Template::NodeSet + + def initialize(parser : Marten::Template::Parser, source : String) + @inner_nodes = parser.parse(up_to: %w(endspaceless)) + parser.shift_token + end + + def render(context : Marten::Template::Context) : String + @inner_nodes.render(context).strip.gsub(/>\s+<") + end +end +``` + +In this example, the `#initialize` method explicitly calls the parser's [`#parse`](pathname:///api/0.3/Marten/Template/Parser.html#parse(up_to%3AArray(String)%3F%3Dnil)%3ANodeSet-instance-method) in order to parse the following "nodes" up to the expected closing tag (`endspaceless` in this case). If the specified closing tag is not encountered, the parser will automatically raise a syntax error. The obtained nodes are returned as a "node set" (instance of [`Marten::Template::NodeSet`](pathname:///api/0.3/Marten/Template/NodeSet.html)): this is a special object returned by the template parser that maps to multiple parsed nodes (those can be tags, variables, or plain text values) that can be rendered through a [`#render`](pathname:///api/0.3/Marten/Template/NodeSet.html#render(context%3AContext)-instance-method) method at rendering time. + +The `#render` method of the above tag is relatively simple: it simply "renders" the node set corresponding to the template nodes that were extracted between the `{% spaceless %}...{% endspaceless %}` tags and then removes any whitespaces between the HTML tags in the output. + +## Registering template tags + +In order to be able to use custom template tags, you must register them to Marten's global template tags registry. + +To do so, you will have to call the [`Marten::Template::Tag#register`](pathname:///api/0.3/Marten/Template/Tag.html#register(tag_name%3AString|Symbol%2Ctag_klass%3ABase.class)-class-method) method with the name of the tag you wish to use in templates, and the template tag class. + +For example: + +```crystal +Marten::Template::Tag.register("local_time", LocalTimeTag) +``` + +With the above registration, you could technically use this tag (the one from the above [Simple tags](#simple-tags) section) as follows: + +```html +{% local_time "%Y-%m-%d %H:%M:%S %:z" %} +``` diff --git a/docs/versioned_docs/version-0.3/templates/introduction.md b/docs/versioned_docs/version-0.3/templates/introduction.md new file mode 100644 index 000000000..080e28b74 --- /dev/null +++ b/docs/versioned_docs/version-0.3/templates/introduction.md @@ -0,0 +1,353 @@ +--- +title: Introduction to templates +description: Learn how to write templates and generate HTML dynamically. +sidebar_label: Introduction +--- + +Templates provide a convenient way for defining the presentation logic of a web application. They allow to write textual content that is rendered dynamically by using a dedicated syntax. This syntax enables the use of dynamic variables as well as some programming constructs. + +## Syntax + +A template is a textual document or a string that makes use of the Marten template language, and that can be used to generate _any_ text-based format (HTML, XML, etc). In order to insert dynamic content, templates usually make use of a few constructs such as **variables**, which are replaced by the corresponding values when the template is evaluated, and **tags**, which can be used to implement the logic of the template. + +For example, the following template displays the properties of an `article` variable and loops over the associated comments in order to display them as a list: + +```html +

{{ article.title }}

+

{{ article.content }}

+
    +{% for comment in article.comments %} +
  • {{ comment.message }}
  • +{% else %} +
  • No comments!
  • +{% endfor %} +
+``` + +Templates need to be evaluated with a **context**. This context is usually a hash-like object containing all the variables or values that can be used by the template when it is rendered. + +In the previous example, the template context would at least contain one `article` key giving access to the considered article properties. + +### Variables + +Variables can be used to inject a value from the context into the rendered template. They must be surrounded by **`{{`** and **`}}`**. + +For example: + +```html +Hello, {{ name }}! +``` + +If the context used to render the above template is `{"name" => "John Doe"}`, then the output would be "Hello, John Doe!". + +Each variable can involve additional lookups in order to access specific object attributes (if such objects have ones). These lookups are expressed by relying on a dot notation (`foo.bar`). For example, the following snippet would output the `title` attribute of the `article` variable: + +```html +

{{ article.title }}

+``` + +This notation can be used to call object methods but also to perform key lookups for hashes or named tuples. It can also be used to perform index lookups for indexable objects (such as arrays or tuples): + +``` +{{ my_array.0 }} +``` + +### Filters + +Filters can be applied to [variables](#variables) or [tag](#tags) arguments in order to transform their values. They are applied to these variables or arguments through the use of a pipe (**`|`**) followed by the name of the filter. + +For example, the following snippet will apply the [`capitalize`](./reference/filters#capitalize) filter to the output of the `name` variable, which will capitalize the value of this variable: + +```html +Hello, {{ name|capitalize }}! +``` + +It should be noted that some filters can take an argument. When this is the case, the argument is specified following a colon character (**`:`**). + +For example, the following snippet will apply the [`default`](./reference/filters#default) filter to the output of the `name` variable in order to fallback to a default name if the variable has a null value: + +```html +Hello, {{ name|default:"Stranger" }}! +``` + +It should be noted that the fact that an argument is supported or not, and mandatory or not, varies based on the considered filter. In all cases, filters can support up to **one** argument only. + +Please head over to the [filters reference](./reference/filters) to see a list of all the available filters. Implementing custom filters is also a possibility that is documented in [Create custom filters](./how-to/create-custom-filters). + +### Tags + +Tags allow to do method-calling and to run any kind of logic within a template. Some tags allow to perform control flows (like if conditions, or for loops) while others simply output values. They are delimited by **`{%`** and **`%}`**. + +For example, the following snippet makes use of the [`assign`](./reference/tags#assign) tag to create a new variable within a template: + +```html +{% assign my_var = "Hello World!" %} +``` + +As mentioned above, some tags allow to perform control flows and require a "closing" tag, like the [`for`](./reference/tags#for) or [`if`](./reference/tags#if) tags: + +```html +{% for article in articles %} + {{ article.title }} is {% if not article.published? %}not {% endif %}published +{% endfor %} +``` + +Some tags also require arguments. For example, the [`url`](./reference/tags#url) template tag requires at least the name of the route for which the URL resolution should be performed: + +```html +{% url "my_route" %} +``` + +Please head over to the [tags reference](./reference/tags) to see a list of all the available template tags. Implementing custom tags is also a possibility that is documented in [Create custom tags](./how-to/create-custom-tags). + +### Comments + +Comments can be inserted in any templates and must be surrounded by **`{#`** and **`#}`**: + +```html +{# This will not be evaluated #} +``` + +## Template inheritance + +Templates can inherit from each other: this allows you to easily define a "base" template containing the layout of your application so that you can reuse it in order to build other templates, which helps in keeping your codebase DRY. + +This works as follows: + +* a "base" template defines the shared layout as well as "blocks" where child templates will actually inject their own contents +* "child" templates "extend" from the base template and explicitly define the contents of the "blocks" that are expected by the base template + +For example, a "base" template could look like this: + +```html + + + {% block title %}My super website{% endblock %} + + + {% block content %}{% endblock %} + + +``` + +Here the base template defines two blocks by using the [`block`](./reference/tags#block) template tag. Using this tag essentially makes it possible for any child templates to "override" the content of these blocks. + +:::tip +Note that it is possible to specify the name of the block being closed in the `endblock` tag to improve readability. For example: + +```html +{% block title %} +My super website +{% endblock title %} +```` +::: + +Given the above base template (that we assume is named `base.html`), a "child" template making use of it could look like this: + +```html +{% extend "base.html" %} + +{% block title %}Custom page title{% endblock %} + +{% block content %}Custom page content{% endblock %} +``` + +Here we make use of the [`extend`](./reference/tags#extend) template tag in order to indicate that we want to inherit from the `base.html` template that we created previously. When Marten encounters this tag, it'll make sure that the targetted template is properly loaded before resuming the evaluation of the current template. + +:::warning +The `{% extend %}` tag should always be called at the top of the file, before the actual content of the template. Inheritance won't work properly if that's not the case. +::: + +We also use [`block`](./reference/tags#block) tags to redefine the content of the blocks that were defined in the `base.html` template. It should be noted that if a child template does not define the content of one of its parent's blocks, the default content of this block will be used instead (if there is one!). + +:::info +You can use many levels of template inheritance if needed. Indeed, a `child.html` template can very well extend a `base_dashboard.html` template, which itself extends a `base.html` template for example. +::: + +It should be noted that it is also possible to get the content of a block from a parent template by using the `super` template tag. This can be useful in situations where blocks in a child template need to extend (add content) to a parent's block content instead of overwriting it. + +For example, with the following snippet the output of the `title` block would be "My super website - Example page": + +```html +{% extend "base.html" %} + +{% block title %}{% super %} - Example page{% endblock %} + +{% block content %}Custom page content{% endblock %} +``` + +It's important to remember that the `super` template tag can only be used within `block` tags. + +## Template loading + +Templates can be loaded from specific locations within your codebase and from application folders. This is controlled by two main settings: + +* [`templates.app_dirs`](../development/reference/settings#app_dirs-1) is a boolean that indicates whether or not it should be possible to load templates that are provided by [installed applications](../development/reference/settings#installed_apps). Indeed, applications can define a `templates` folder at their root, and these templates will be discoverable by Marten if this setting is set to `true` +* [`templates.dirs`](../development/reference/settings#dirs1) is an array of additional directories where templates should be looked for + +Application templates are always enabled by default (`templates.app_dirs = true`) for new Marten projects. + +It is possible to programmatically load a template by name. To do so, you can use the [`#get_template`](pathname:///api/0.3/Marten/Template/Engine.html#get_template(template_name%3AString)%3ATemplate-instance-method) method that is provided by the Marten templates engine: + +```crystal +Marten.templates.get_template("foo/bar.html") +``` + +This will return a compiled [`Template`](pathname:///api/0.3/Marten/Template/Template.html) object that you can then render by using a specific context. + +## Rendering a template + +You won't usually need to interact with the "low-level" API of the Marten template engine in order to render templates: most of the time you will render templates as part of [handlers](../handlers-and-http), which means that you will likely end up using the [`#render`](../handlers-and-http/introduction#render) shortcut or [generic handlers](../handlers-and-http/generic-handlers) that automatically render templates for you. + +That being said, it is also possible to render any [`Template`](pathname:///api/0.3/Marten/Template/Template.html) object that you loaded by leveraging the [`#render`](pathname:///api/0.3/Marten/Template/Template.html#render(context%3AHash|NamedTuple)%3AString-instance-method) method. This method can be used either with a Marten context object, a hash, or a named tuple: + +```crystal +template = Marten.templates.get_template("foo/bar.html") +template.render(Marten::Template::Context{"foo" => "bar"}) +template.render({"foo" => "bar"}) +template.render({ foo: "bar" }) +``` + +## Using custom objects in contexts + +Most objects that are provided by Marten (such as Model records, query sets, schemas, etc) can automatically be used as part of templates. If your project involves other custom classes, and if you would like to interact with such objects in your templates, then you will need to explicitly ensure that they include the [`Marten::Template::Object`](pathname:///api/0.3/Marten/Template/Object.html) module. + +:::note Why? +Crystal being a statically typed language, the Marten engine needs to know which types of objects it is dealing with in advance in order to know (i) what can go into template contexts and (ii) how to "resolve" object attributes when templates are rendered. It is not possible to simply expect any `Object` object, hence why we need to make use of a shared [`Marten::Template::Object`](pathname:///api/0.3/Marten/Template/Object.html) module to account for all the classes whose objects should be usable as part of template contexts. +::: + +Let's take the example of a `Point` class that provides access to an x-coordinate and a y-coordinate: + +```crystal +class Point + getter x + getter y + + def initialize(@x : Int32, @y : Int32) + end +end +``` + +By default, `Point` objects cannot be used as part of templates. Let's say we want to render the following template involving a `point` variable: + +```html +My point is: {{ point.x }}, {{ point.y }} +``` + +If you try to render such a template while passing a `Point` object into the template context, you will encounter a `Marten::Template::Errors::UnsupportedValue` exception stating: + +``` +Unable to initialize template values from Point objects +``` + +To remediate this, you will have to include the [`Marten::Template::Object`](pathname:///api/0.3/Marten/Template/Object.html) module in the `Point` class and define a `#resolve_template_attribute` method as follows: + +```crystal +class Point + include Marten::Template::Object + + getter x + getter y + + def initialize(@x : Int32, @y : Int32) + end + + def resolve_template_attribute(key : String) + case key + when "x" + x + when "y" + y + end + end +end +``` + +Each class including the [`Marten::Template::Object`](pathname:///api/0.3/Marten/Template/Object.html) module must also implement a `#resolve_template_attribute` method in order to allow resolutions of object attributes when templates are rendered (for example `{{ point.x }}`). That being said, there are a few shortcuts that can be used in order to avoid writing such methods. + +The first one is to use the [`#template_attributes`](pathname:///api/0.3/Marten/Template/Object.html#template_attributes(*names)-macro) macro in order to easily define the names of the methods that should be made available to the template runtime. For example, such macro could be used like this with our `Point` class: + +```crystal +class Point + include Marten::Template::Object + + getter x + getter y + + def initialize(@x : Int32, @y : Int32) + end + + template_attributes :x, :y +end +``` + +Another possibility is to include the [`Marten::Template::Object::Auto`](pathname:///api/0.3/Marten/Template/Object/Auto.html) module instead of the [`Marten::Template::Object`](pathname:///api/0.3/Marten/Template/Object.html) one in your class. This module will automatically ensure that every "attribute-like" public method that is defined in the including class can also be accessed in templates when performing variable lookups. + +```crystal +class Point + include Marten::Template::Object::Auto + + getter x + getter y + + def initialize(@x : Int32, @y : Int32) + end +end +``` + +Note that **all** "attribute-like" public methods will be made available to the template runtime when using the [`Marten::Template::Object::Auto`](pathname:///api/0.3/Marten/Template/Object/Auto.html) module. This may be a good enough behavior, but if you want to have more control over what can be accessed in templates or not, you will likely end up using [`Marten::Template::Object`](pathname:///api/0.3/Marten/Template/Object.html) and the [`#template_attributes`](pathname:///api/0.3/Marten/Template/Object.html#template_attributes(*names)-macro) macro instead. + +## Using context producers + +Context producers are helpers that ensure that common variables are automatically inserted in the template context whenever a template is rendered. They are applied every time a new template context is generated. + +For example, they can be used to insert the current HTTP request object in every template context being rendered in the context of a handler and HTTP request. This makes sense considering that the HTTP request object is a common object that is likely to be used by multiple templates in your project: that way there is no need to explicitly "insert" it in the context every time you render a template. This specific capability is provided by the [`Marten::Template::ContextProducer::Request`](pathname:///api/0.3/Marten/Template/ContextProducer/Request.html) context producer, which inserts a `request` object into every template context. + +Template context producers can be configured through the use of the [`templates.context_producers`](../development/reference/settings#contextproducers) setting. When generating a new project by using the `marten new` command, the following context producers will be automatically configured: + +```crystal +config.templates.context_producers = [ + Marten::Template::ContextProducer::Request, + Marten::Template::ContextProducer::Flash, + Marten::Template::ContextProducer::Debug, + Marten::Template::ContextProducer::I18n, +] +``` + +Each context producer in this array will be applied in order when a new template context is created and will contribute "common" context values to it. This means that the order of these is important since context producers can technically overwrite the values that were added by previous context producers. + +Please head over to the [context producers reference](./reference/context-producers) to see a list of all the available context producers. Implementing custom context producers is also a possibility that is documented in [Create custom context producers](./how-to/create-custom-context-producers). + +## Auto-escaping + +The output of template variables is automatically escaped by Marten in order to prevent Cross-Site Scripting (XSS) vulnerabilities. + +For example, let's consider the following snippet: + +```html +Hello, {{ name }}! +``` + +If this template is rendered with `` as the content of the `name` variable, then the output will be: + +```html +Hello, <script>alert('popup')</script>! +``` + +It should be noted that this behavior can be disabled _explicitly_. Indeed, sometimes it is expected that some template variables will contain trusted HTML content that you intend to embed into the template's HTML. + +To do this, it is possible to make use of the [`safe`](./reference/filters#safe) template filter. This filter "marks" the output of a variable as safe, which ensures that its content is not escaped before being inserted in the final output of a rendered template. + +For example: + +```html +Hello, {{ name }}! +Hello, {{ name|safe }}! +``` + +When rendered with `John` as the content of the `name` variable, the above template will output: + +```html +Hello, <b>John</b>! +Hello, John! +``` diff --git a/docs/versioned_docs/version-0.3/templates/reference/context-producers.md b/docs/versioned_docs/version-0.3/templates/reference/context-producers.md new file mode 100644 index 000000000..ed64910fd --- /dev/null +++ b/docs/versioned_docs/version-0.3/templates/reference/context-producers.md @@ -0,0 +1,34 @@ +--- +title: Context producers +description: Context producers reference. +--- + +This page provides a reference for all the available context producers that can be used when rendering [templates](../introduction). + +## Debug context producer + +**Class:** [`Marten::Template::ContextProducer::Debug`](pathname:///api/0.3/Marten/Template/ContextProducer/Debug.html) + +The Debug context producer contributes a `debug` variable to the context: the associated value is `true` or `false` depending on whether [debug mode](../../development/reference/settings#debug) is enabled for the project or not. + +## Flash context producer + +**Class:** [`Marten::Template::ContextProducer::Flash`](pathname:///api/0.3/Marten/Template/ContextProducer/Flash.html) + +The Flash context producer contributes a `flash` variable to the context: this variable corresponds to the [flash store](../../handlers-and-http/introduction#using-the-flash-store) that is associated with the current HTTP request. If the template context is not initialized with an HTTP request object, then no variables are inserted. + +## I18n context producer + +**Class:** [`Marten::Template::ContextProducer::I18n`](pathname:///api/0.3/Marten/Template/ContextProducer/I18n.html) + +The I18n context producer contributes I18n-related variables to the context: + +* `locale`: the current locale +* `available_locales`: an array of all the available locales that can be activated for the project + +## Request context producer + +**Class:** [`Marten::Template::ContextProducer::Request`](pathname:///api/0.3/Marten/Template/ContextProducer/Request.html) + +The Request context producer contributes a `request` variable to the context: this variable corresponds to the current HTTP request object. If the template context is not initialized with an HTTP request object, then no variables are inserted. + diff --git a/docs/versioned_docs/version-0.3/templates/reference/filters.md b/docs/versioned_docs/version-0.3/templates/reference/filters.md new file mode 100644 index 000000000..932d2cf96 --- /dev/null +++ b/docs/versioned_docs/version-0.3/templates/reference/filters.md @@ -0,0 +1,114 @@ +--- +title: Template filters +description: Template filters reference. +--- + +This page provides a reference for all the available filters that can be used when defining [templates](../introduction). + +## `capitalize` + +The `capitalize` filter allows to modify a string so that the first letter is converted to uppercase and all the subsequent letters are converted to lowercase. + +For example: + +```html +{{ value|capitalize }} +``` + +If `value` is "marten", the output will be "Marten". + +## `default` + +The `default` filter allows to fallback to a specific value if the left side of the filter expression is not truthy. A filter argument is mandatory. It should be noted that empty strings are considered truthy and will be returned by this filter. + +For example: + +```html +{{ value|default:"foobar" }} +``` + +If `value` is `nil` (or `0` or `false`), the output will be "foobar". + +## `downcase` + +The `downcase` filter allows to convert a string so that each of its characters is lowercase. + +For example: + +```html +{{ value|downcase }} +``` + +If `value` is "Hello", then the output will be "hello". + +## `join` + +The `join` filter converts an array of elements into a string separated by `arg`. + +For example: + +```html +{{ value|join: arg }} +``` + +If `value` is `["Bananas","Apples","Oranges"]` and `arg` is `, `, then the output will be "Bananas, Apples, Oranges". + +## `linebreaks` + +The `linebreaks` filter allows to convert a string replacing all newlines with HTML line breaks (`
`). + +For example: + +```html +{{ value|linebreaks }} +``` + +If `value` is `Hello\nWorld`, then the output will be `Hello
World`. + +## `safe` + +The `safe` filter allows to mark that a string is safe and that it should not be escaped before being inserted in the final output of a rendered template. Indeed, string values are always automatically HTML-escaped by default in templates. + +For example: + +```html +{{ value|safe }} +``` + +If `value` is `

Hello

`, then the output will be `

Hello

` as well. + +## `size` + +The `size` filter allows returning the size of a string or an enumerable object. + +For example: + +```html +{{ value|size }} +``` + +If `value` is `hello`, then the output will be 5. + +## `split` + +The `split` filter converts a string into an array of elements separated by `arg`. + +For example: + +```html +{{ value|split: arg }} +``` + +If `value` is `Bananas,Apples,Oranges` and `arg` is `,`, then the output will be ["Bananas","Apples","Oranges"]. + +## `upcase` + +The `upcase` filter allows to convert a string so that each of its characters is uppercase. + +For example: + +```html +{{ value|upcase }} +``` + +If `value` is "Hello", then the output will be "HELLO". diff --git a/docs/versioned_docs/version-0.3/templates/reference/tags.md b/docs/versioned_docs/version-0.3/templates/reference/tags.md new file mode 100644 index 000000000..3e386b067 --- /dev/null +++ b/docs/versioned_docs/version-0.3/templates/reference/tags.md @@ -0,0 +1,259 @@ +--- +title: Template tags +description: Template tags reference. +--- + +## `asset` + +The `asset` template tag allows to generate the URL of a given [asset](../../assets/introduction). It must take at least one argument (the filepath of the asset). + +For example, the following line is a valid usage of the `asset` tag and will output the path or URL of the `app/app.css` asset: + +```html +{% asset "app/app.css" %} +``` + +Optionally, resolved asset URLs can be assigned to a specific variable using the `as` keyword: + +```html +{% asset "app/app.css" as my_var %} +``` + +## `assign` + +The `assign` template tag allows to define a new variable that will be stored in the template's context. + +For example: + +```html +{% assign my_var = "Hello World!" %} +``` + +## `block` + +The `block` template tag allows to define that some specific portions of a template can be overridden by child templates. This tag is only useful when used in conjunction with the [`extend`](#extend) tag. See [Template inheritance](../introduction#template-inheritance) to learn more about this capability. + +## `cache` + +The `cache` template tag allows to cache the content of a template fragment (enclosed within the `{% cache %}...{% endcache %}` tags) for a specific duration. This caching operation is done by leveraging the configured [cache store](../../caching/introduction#configuration-and-cache-stores). + +At least a cache key and and a cache expiry (expressed in seconds) must be specified when using this tag: + +```html +{% cache "mykey" 3600 %} + Cached content! +{% endcache %} +``` + +It should be noted that the `cache` template tag also supports specifying additional "vary on" arguments that allow to invalidate the cache based on the value of other template variables: + +```html +{% cache "mykey" 3600 current_locale user.id %} + Cached content! +{% endcache %} +``` + +## `csrf_token` + +The `csrf_token` template tag allows to compute and insert the value of the CSRF token into a template. This tag can only be used for templates that are rendered as part of a handler (for example by leveraging [`#render`](../../handlers-and-http/introduction#render) or one of the [generic handlers](../../handlers-and-http/generic-handlers) involving rendered templates). + +This can be used to insert the CSRF token into a hidden form input so that it gets sent to the handler processing the form data for example. Indeed, handlers will automatically perform a CSRF check in order to protect unsafe requests (ie. requests whose methods are not `GET`, `HEAD`, `OPTIONS`, or `TRACE`): + +```html +
+ + + +
+``` + +See [Cross-Site Request Forgery protection](../../security/csrf) to learn more about this. + +## `extend` + +The `extend` template tag allows to define that a template inherits from a specific base template. This tag must be used with one mandatory argument, which can be either a string literal or a variable that will be resolved at runtime. This mechanism is useful only if the base template defines [blocks](#block) that are overridden or extended by the child template. See [Template inheritance](../introduction#template-inheritance) to learn more about this capability. + +## `for` + +The `for` template tag allows to loop over the items of iterable objects and it also handles fallbacks through the use of the `else` inner block. It should be noted that the `for` template tag requires a closing `endfor` tag. + +For example: + +```html +{% for item in items %} + Display {{ item }} +{% else %} + No items! +{% endfor %} +``` + +It should be noted that `for` loops support unpacking multiple items when applicable (eg. when iterating over hashes or enumerables containing arrays or tuples): + +```html +{% for label, url in navigation_items %} + {{ label }} +{% endfor %} +``` + +Finally, loops give access to a special `loop` variable _inside_ the loop in order to expose information about the iteration process: + +| Variable | Description | +| -------- | ----------- | +| `loop.index` | The index of the current iteration (1-indexed) | +| `loop.index0` | The index of the current iteration (0-indexed) | +| `loop.revindex` | The index of the current iteration counting from the end of the loop (1-indexed) | +| `loop.revindex0` | The index of the current iteration counting from the end of the loop (0-indexed) | +| `loop.first?` | A boolean indicating if this is the first iteration of the loop | +| `loop.last?` | A boolean indicating if this is the last iteration of the loop | +| `loop.parent` | The parent's `loop` variable (only for nested for loops) | + +## `if` + +The `if` template tags allows to define conditions allowing to control which blocks should be executed. An `if` tag must always start with an `if` condition, followed by any number of intermediate `elsif` conditions and an optional (and final) `else` block. It also requires a closing `endif` tag. + +For example: + +```html +{% if my_var == 0 %} + Zero! +{% elsif my_var == 1 %} + One! +{% else %} + Other! +{% endif %} +``` + +## `include` + +The `include` template tag allows to include and render another template using the current context. This tag must be used with one mandatory argument: the name of the template to include, which can be either a string literal or a variable that will be resolved at runtime. + +For example: + +```html +{% include "path/to/my_snippet.html" %} +``` + +Included templates are rendered using the context of the including template. This means that all the variables that are expected or provided to the including template can also be used as part of the included template. + +For example: + +```html title="hello.html" +Hello, {{ name }}! {% include "question.html" %} +``` + +```html title="question.html" +How are you {{ name }}? +``` + +If `name` is "John", then the output will be "Hello, John! How are you John?". + +Finally, it should be noted that additional variables that are specific to the included template only can be specified using the `with` keyword: + +```html +{% include "path/to/my_snippet.html" with new_var="hello" %} +``` + +:::caution +Templates that are included using the `include` template are parsed and rendered _when_ the including template is rendered as well. Included templates are not parsed when the including template is parsed itself. This means that the including template and the included template are always rendered _separately_. +::: + +## `local_time` + +The `local_time` template tag allows to output the string representation of the local time. It must take one argument (the [format](https://crystal-lang.org/api/Time/Format.html) used to output the time). + +For example, the following lines are valid usages of the `local_time` tag: + +```html +{% local_time "%Y" %} +{% local_time "%Y-%m-%d %H:%M:%S %:z" %} +``` + +Optionally, the output of this tag can be assigned to a specific variable using the `as` keyword: + +```html +{% local_time "%Y" as current_year %} +``` + +## `spaceless` + +The `spaceless` template tag allows to remove whitespaces, tabs, and new lines between HTML tags. Whitespaces inside tags are left untouched. It should be noted that the `spaceless` template tag requires a closing `endspaceless` tag. + +For example: + +```html +{% spaceless %} +

+ Sign In +

+{% endspaceless %} +``` + +Would output the following: + +```html +

Sign In

+``` + +## `super` + +The `super` template tag allows to render the content of a block from a parent template (in a situation where both the `extend` and `block` tags are used). This can be useful in situations where blocks in a child template need to extend (add content) to a parent's block content instead of overwriting it. See [Template inheritance](../introduction#template-inheritance) to learn more about this capability. + +## `translate` + +The `translate` template tag allows to perform translation lookups by using the [I18n configuration](../../development/reference/settings#i18n-settings) of the project. It must take at least one argument (the translation key) followed by keyword arguments. + +For example the following lines are valid usages of the `translate` tag: + +```html +{% translate "simple.translation" %} +{% translate "simple.interpolation" value: 'test' %} +``` + +Translation keys and parameter values can be resolved as template variables too, but they can also be defined as literal values if necessary. + +Optionally, resolved translations can be assigned to a specific variable using the `as` keyword: + +```html +{% translate "simple.interpolation" value: 'test' as my_var %} +``` + +## `trans` + +Alias for [`translate`](#translate). + +## `t` + +Alias for [`translate`](#translate). + +## `url` + +The `url` template tag allows to perform [URL lookups](../../handlers-and-http/routing#reverse-url-resolutions). It must take at least one argument (the name of the targeted handler) followed by optional keyword arguments (if the route requires parameters). + +For example, the following lines are valid usages of the `url` tag: + +```html +{% url "my_handler" %} +{% url "my_other_handler" arg1: var1, arg2: var2 %} +``` + +URL names and parameter values can be resolved as template variables too, but they can also be defined as literal values if necessary. + +Optionally, resolved URLs can be assigned to a specific variable using the `as` keyword: + +```html +{% url "my_other_handler" arg1: var1, arg2: var2 as my_var %} +``` + +## `verbatim` + +The `verbatim` template tag prevents the content of the tag to be processed by the template engine. It should be noted that the `verbatim` template tag requires a closing `endverbatim` tag. + +For example: + +``` +{% verbatim %} + This should not be {{ processed }}. +{% endverbaim %} +``` + +Would output `This should not be {{ processed }}.`. diff --git a/docs/versioned_docs/version-0.3/the-marten-project.mdx b/docs/versioned_docs/version-0.3/the-marten-project.mdx new file mode 100644 index 000000000..383cf00ee --- /dev/null +++ b/docs/versioned_docs/version-0.3/the-marten-project.mdx @@ -0,0 +1,24 @@ +--- +title: The Marten project +--- + +import DocCard from '@theme/DocCard'; + +This section provides general information about the Marten project itself and details how to get involved and contribute to the framework. + +## Guides + +
+
+ +
+
+ +
+
+ +
+
+ +
+
diff --git a/docs/versioned_docs/version-0.3/the-marten-project/acknowledgments.md b/docs/versioned_docs/version-0.3/the-marten-project/acknowledgments.md new file mode 100644 index 000000000..1b80d0619 --- /dev/null +++ b/docs/versioned_docs/version-0.3/the-marten-project/acknowledgments.md @@ -0,0 +1,40 @@ +--- +title: Acknowledgements +description: Thanks and attributions related to the projects and concepts that inspired the Marten web framework. +--- + +This section lists and acknowledges the various projects that inspired the Marten web framework, as well as notable contributions. + +## Inspirations + +The Marten web framework implements a set of ideas and APIs that are inspired by the awesome work that was put into two particular frameworks: [Django](https://www.djangoproject.com/) and [Ruby on Rails](https://rubyonrails.org/). It can be easy to take for granted what these frameworks provide, and we should not forget to give credit where credit is due. + +### Django + +The Marten web framework takes a lot from [Django](https://www.djangoproject.com/) - its biggest source of inspiration: + +* The [Model](../models-and-databases)-[Handler](../handlers-and-http)-[Template](../templates) triptych is Marten's vision of the Model-View-Template (MVT) pattern provided by Django +* The [auto-generated migrations](../models-and-databases/migrations) mechanism is inspired by a similar mechanism within Django +* [Generic handlers](../handlers-and-http/generic-handlers) are inspired by Django's generic class-based views +* The template syntax is inspired by Django's templating language +* The concept of [apps](../development/applications) and projects is inherited from Django as well + +Needless to say that this is a non-exhaustive list. + +### Ruby on Rails + +The Marten web framework is also inspired by [Ruby on Rails](https://rubyonrails.org/) in some aspects. Among those we can mention: + +* The [generic validation DSL](../models-and-databases/validations) +* Most [model callbacks](../models-and-databases/callbacks) +* The idea of [message encryptors](pathname:///api/0.3/Marten/Core/Encryptor.html) and [message signers](pathname:///api/0.3/Marten/Core/Signer.html) + +### But also... + +* The exception page displayed while in [debug](../development/reference/settings#debug) mode is inspired by the [Exception Page](https://github.com/crystal-loot/exception_page) shard +* The way to [handle custom objects](../templates/introduction#using-custom-objects-in-contexts) in Marten templates is inspired by a similar mechanism within [Crinja](https://github.com/straight-shoota/crinja) +* The idea of class-based [email definitions](../emailing/introduction) is borrowed from [Carbon](https://github.com/luckyframework/carbon) + +## Contributors + +Thanks to all the [contributors](https://github.com/martenframework/marten/contributors) of the project! diff --git a/docs/versioned_docs/version-0.3/the-marten-project/contributing.md b/docs/versioned_docs/version-0.3/the-marten-project/contributing.md new file mode 100644 index 000000000..3b4fc475b --- /dev/null +++ b/docs/versioned_docs/version-0.3/the-marten-project/contributing.md @@ -0,0 +1,115 @@ +--- +title: Contributing to the Marten project +description: Learn how you can start contributing to the Marten project. +sidebar_label: Contributing +--- + +Marten is a big project that will keep evolving over time. If you want to, there are many ways to get involved! + +## First things first... + +The Marten project adheres to the [Contributor Covenant Code of Conduct](https://github.com/martenframework/marten/blob/main/CODE_OF_CONDUCT.md). Everyone contributing to the Marten project is expected to follow this code. + +## Reporting issues + +You should use the [project's issue tracker](https://github.com/martenframework/marten/issues) that is hosted on GitHub if you've found a bug or if you want to propose a new feature. + +### About bug reports + +If you've found a bug related to the Marten web framework **that is not a security issue**, then you should (i) search the [existing issues](https://github.com/martenframework/marten/issues) to verify that it hasn't been reported yet and (ii) [create a new issue](https://github.com/martenframework/marten/issues/new) if that's not the case. Don't forget to include as many details as possible in your tickets: an explanatory description of the issue at hand and how to reproduce it, snippets and/or tracebacks if this is appropriate, etc. + +### About security issues + +If you've found a security issue please **do not open a GitHub issue**. Instead, send an email to `security@martenframework.com`. We'll then investigate together to resolve the problem so we can make an announcement about a solution along with the vulnerability. + +## Contributing code + +The preferred way to contribute to the Marten framework is to submit pull requests to the project's [GitHub repository](https://github.com/martenframework/marten). If you don't know where to start and would like to start contributing code to the Marten framework, you can have a look at the [Good first issues](https://github.com/martenframework/marten/issues?q=is%3Aissue+is%3Aopen+label%3A%22Good+first+issue%22). + +Below are provided some general tips regarding contributing code: development environments, running tests, etc. + +### Development environment + +:::info +The following steps assume that you have at least [git](https://git-scm.com/), [Crystal](https://crystal-lang.org/), and [Node.js](https://nodejs.org) installed on your system. +::: + +First, you should [fork](https://github.com/martenframework/marten/fork) the Marten git repository. Then you can get a local copy of the project using the following command: + +```bash +git clone git@github.com:/marten.git +``` + +Once this is done you should change into the `marten` repository and install the framework dependencies by running the following command: + +```bash +make +``` + +This will install a bunch of Crystal shards and some Node.js dependencies (which are required to work on the [Docusaurus](https://docusaurus.io/)-powered documentation). + +### Coding style + +Overall, Marten tries to comply with Crystal's [style guide](https://crystal-lang.org/reference/conventions/coding_style.html) and you should ensure that your changes comply with it as well if you are contributing code to the project. + +In addition to that, Marten's codebase is checked using [ameba](https://github.com/crystal-ameba/ameba) and via the standard [`crystal tool format`](https://crystal-lang.org/reference/man/crystal/index.html#crystal-tool-format) command. Every pull request opened on [Marten's GitHub repository](https://github.com/martenframework/marten) will be checked using these tools automatically. If needed, you can verify that these checks are passing locally by running the following commands: + +```bash +make qa # Run both ameba and the Crystal formatting checks +make lint # Run ameba checks only +make format_checks # Run Crystal formatting checks only +``` + +Additionally, you can also apply Crystal's default formatting to the codebase by running: + +```bash +make format +``` + +### Tests + +You should not submit pull requests without providing tests. Marten uses the standard [spec module](https://crystal-lang.org/reference/guides/testing.html) and comes with an extensive spec suite. + +Specs must be defined in the `spec` folder, at the root of the project's repository. You can run the whole spec suite by using the following command (which is equivalent to running `crystal spec`): + +```bash +make tests +``` + +By default, specs will be executed using an in-memory SQLite database. If you wish to, you can configure additional databases by updating the `.spec.env.json` file that should've been automatically generated by the `make` command earlier (cf. [Development environment](#development-environment)). This file defines a bunch of "environment" setting values that are automatically leveraged to configure the test project that is used when running specs. It looks something like this: + +```json title=.spec.env.json +{ + "MYSQL_DEFAULT_DB_NAME": "example", + "MYSQL_OTHER_DB_NAME": "other_example", + "MYSQL_DB_USER": "example", + "MYSQL_DB_PASSWORD": "", + "MYSQL_DB_HOST": "", + "POSTGRESQL_DEFAULT_DB_NAME": "example", + "POSTGRESQL_OTHER_DB_NAME": "other_example", + "POSTGRESQL_DB_USER": "example", + "POSTGRESQL_DB_PASSWORD": "", + "POSTGRESQL_DB_HOST": "" +} +``` + +As you can see, you have to specify two databases for each database backend (MySQL and PostgreSQL). This is mandatory because Marten's specs are also testing cases where multiple databases are configured and used simultaneously for the same project. + +Specs are always executed using a _single_ database backend. As mentioned previously this database backend is the SQLite one by default, but you can specify the one to use when running specs by setting the `MARTEN_SPEC_DB_CONNECTION` environment variable. For example: + +```bash +MARTEN_SPEC_DB_CONNECTION=mysql make tests # Will run specs using the MySQL DB backend +MARTEN_SPEC_DB_CONNECTION=postgresql make tests # Will run specs using the PostgreSQL DB backend +``` + +### Documentation + +Marten's documentation is written using Markdown. It is powered by [Docusaurus](https://docusaurus.io/) and lives under the `docs` folder. + +To run the documentation live server locally, you can change into `docs` and make use of the following command: + +```bash +npm run start +``` + +This will start the Docusaurus server at [http://localhost:3000/docs/](http://localhost:3000/docs/) and you will be able to easily see and test the changes you are making to the documentation source files. diff --git a/docs/versioned_docs/version-0.3/the-marten-project/design-philosophies.md b/docs/versioned_docs/version-0.3/the-marten-project/design-philosophies.md new file mode 100644 index 000000000..5f97d1f66 --- /dev/null +++ b/docs/versioned_docs/version-0.3/the-marten-project/design-philosophies.md @@ -0,0 +1,34 @@ +--- +title: Design philosophies +description: Learn about the design philosophies behind the Marten web framework. +--- + +This document goes over the fundamental design philosophies that influenced the creation of the Marten web framework. It seeks to provide insight into the past and serve as a reference for the future. + +## Simple and easy to use + +Marten tries to ensure that everything it enables is as simple as possible and that the syntax provided for dealing with the framework's components remains obvious and easy to remember (and certainly not complex or obscure). The framework makes it as easy as possible to leverage its capabilities and perform CRUD operations. + +## Full-featured + +Marten adheres to the "batteries included" philosophy. Out of the box, it provides the tools and features that are commonly required by web applications: [ORM](../models-and-databases/introduction), [migrations](../models-and-databases/migrations), [translations](../i18n/introduction), [templating engine](../templates/introduction), [sessions](../handlers-and-http/sessions), [emailing](../emailing/introduction), and [authentication](../authentication/introduction). + +## Extensible + +Marten gives developers the ability to contribute extra functionalities to the framework easily. Things like [custom model field implementations](../models-and-databases/how-to/create-custom-model-fields), [new route parameter types](../handlers-and-http/how-to/create-custom-route-parameters.md), [session stores](../handlers-and-http/sessions#session-stores), etc... can all be registered to the framework easily. + +## DB-Neutral + +The framework's ORM is and should remain usable with multiple database backends (including MySQL, PostgreSQL, and SQLite). + +## App-oriented + +Marten allows separating projects into a set of logical "[apps](../development/applications)", which helps improve code organization and makes it easy for multiple developers to work on different components. Each app can contribute specific abstractions and features to a project like models and migrations, templates, HTTP handlers and routes, etc. These apps can also be extracted in Crystal shards in order to contribute features and behaviors to other Marten projects. The goal behind this capability is to allow the creation of a powerful apps ecosystem over time and to encourage "reusability" and "pluggability". + +:::tip +In this light, the [Awesome Marten](https://github.com/martenframework/awesome-marten) repository lists applications that you can leverage in your projects. +::: + +## Backend-oriented + +The framework is intentionally very "backend-oriented" because the idea is to not make too many assumptions regarding how the frontend code and assets should be structured, packaged or bundled together. The framework can't account for all the ways assets can be packaged and/or bundled together and does not advocate for specific solutions in this area. Some projects might require a webpack strategy to bundle assets, some might require a fingerprinting step on top of that, and others might need something entirely different. How these toolchains are configured or set up is left to the discretion of web application developers, and the framework simply makes it easy to [reference these assets](../assets/introduction) and [collect them](../assets/introduction#serving-assets-in-production) at deploy time to upload them to their final destination. diff --git a/docs/versioned_docs/version-0.3/the-marten-project/release-notes.md b/docs/versioned_docs/version-0.3/the-marten-project/release-notes.md new file mode 100644 index 000000000..39cbd392a --- /dev/null +++ b/docs/versioned_docs/version-0.3/the-marten-project/release-notes.md @@ -0,0 +1,28 @@ +--- +title: Release notes +description: Find all the release notes of the Marten web framework. +pagination_next: null +--- + +Here are listed the release notes for each version of the Marten web framework. + +## Marten 0.3 + +* [Marten 0.3 release notes](./release-notes/0.3) + +## Marten 0.2 + +* [Marten 0.2.4 release notes](./release-notes/0.2.4) +* [Marten 0.2.3 release notes](./release-notes/0.2.3) +* [Marten 0.2.2 release notes](./release-notes/0.2.2) +* [Marten 0.2.1 release notes](./release-notes/0.2.1) +* [Marten 0.2 release notes](./release-notes/0.2) + +## Marten 0.1 + +* [Marten 0.1.5 release notes](./release-notes/0.1.5) +* [Marten 0.1.4 release notes](./release-notes/0.1.4) +* [Marten 0.1.3 release notes](./release-notes/0.1.3) +* [Marten 0.1.2 release notes](./release-notes/0.1.2) +* [Marten 0.1.1 release notes](./release-notes/0.1.1) +* [Marten 0.1 release notes](./release-notes/0.1) diff --git a/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.1.1.md b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.1.1.md new file mode 100644 index 000000000..e0aa1f1e8 --- /dev/null +++ b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.1.1.md @@ -0,0 +1,12 @@ +--- +title: Marten 0.1.1 release notes +pagination_prev: null +pagination_next: null +--- + +_October 25, 2022._ + +## Bug fixes + +* Fix a possible shard installation issue with older Crystal versions +* The `new` command now properly generates `shard.yml` files that target the latest stable version of Marten instead of the development one diff --git a/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.1.2.md b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.1.2.md new file mode 100644 index 000000000..6e352ae19 --- /dev/null +++ b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.1.2.md @@ -0,0 +1,11 @@ +--- +title: Marten 0.1.2 release notes +pagination_prev: null +pagination_next: null +--- + +_November 6, 2022._ + +## Bug fixes + +* Fix missing multicolumns indexes in migrations generated for newly created models diff --git a/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.1.3.md b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.1.3.md new file mode 100644 index 000000000..87dd78eaa --- /dev/null +++ b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.1.3.md @@ -0,0 +1,14 @@ +--- +title: Marten 0.1.3 release notes +pagination_prev: null +pagination_next: null +--- + +_November 21, 2022._ + +## Bug fixes + +* Ensure that before/after deletion callbacks are executed in the same transaction as the deletion runner +* Add missing `.gitignore` file to projects generated through the use of the [`new`](../../development/reference/management-commands#new) management command +* Fix incorrect host used for development environments when generating new projects through the use of the [`new`](../../development/reference/management-commands#new) management command: now, projects automatically use the `127.0.0.1` local host in development +* Ensure that the [flash context producer](../../templates/reference/context-producers#flash-context-producer) takes into account requests that don't have an associated flash store in order to avoid unexpected `NilAssertionError` exceptions diff --git a/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.1.4.md b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.1.4.md new file mode 100644 index 000000000..16fae917e --- /dev/null +++ b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.1.4.md @@ -0,0 +1,13 @@ +--- +title: Marten 0.1.4 release notes +pagination_prev: null +pagination_next: null +--- + +_December 30, 2022._ + +## Bug fixes + +* Fix non-working `#pk` method generation for models with a one-to-one primary key field +* Fix a possible `NilAssertionError` when running a migration involving a column change +* Fix a possible layout issue in the server error debug page diff --git a/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.1.5.md b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.1.5.md new file mode 100644 index 000000000..89c7dca5f --- /dev/null +++ b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.1.5.md @@ -0,0 +1,14 @@ +--- +title: Marten 0.1.5 release notes +pagination_prev: null +pagination_next: null +--- + +_January 7, 2023._ + +## Bug fixes + +* Fix a bug where some types returned by built-in fields weren't allowed as template values +* Fix a bug where it was not possible to resolve UUID routing parameters from valid UUID strings +* Ensure abstract model classes can be created without primary key fields +* Ensure subclasses of [`Marten::Conf::Setting`](pathname:///api/0.3/Marten/Conf/Settings.html) no longer have to define a mandatory `#initialize` method if they don't need to diff --git a/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.1.md b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.1.md new file mode 100644 index 000000000..35ff823f7 --- /dev/null +++ b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.1.md @@ -0,0 +1,15 @@ +--- +title: Marten 0.1 release notes +pagination_prev: null +pagination_next: null +--- + +_October 23, 2022._ + +## Requirements and compatibility + +Crystal 1.4, 1.5, and 1.6. + +## New features + +_This is the initial release of the Marten web framework!_ diff --git a/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.2.1.md b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.2.1.md new file mode 100644 index 000000000..67be5b3f8 --- /dev/null +++ b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.2.1.md @@ -0,0 +1,13 @@ +--- +title: Marten 0.2.1 release notes +pagination_prev: null +pagination_next: null +--- + +_February 25, 2023._ + +## Bug fixes + +* Fix possible empty field names in messages associated with exceptions raised when targetting non-existing fields with query sets +* Fix broken methods inherited from the [`Enumerable`](https://crystal-lang.org/api/1.6.2/Enumerable.html) mixin for request parameter abstractions +* Fix a possible typing issue related to the use of `Marten::Emailing::Backend::Development#delivered_emails` in specs generated for projects with authentication diff --git a/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.2.2.md b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.2.2.md new file mode 100644 index 000000000..1410d2d43 --- /dev/null +++ b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.2.2.md @@ -0,0 +1,13 @@ +--- +title: Marten 0.2.2 release notes +pagination_prev: null +pagination_next: null +--- + +_March 7, 2023._ + +## Bug fixes + +* Fix possible compilation errors happening when inheriting from a schema containing fields with options +* Fix an issue where model field validations were inconsistent with validation callbacks (eg. model field validations were called before `before_validation` callbacks) +* Ensure that exception messages and snippet lines are properly escaped in the server error debug page diff --git a/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.2.3.md b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.2.3.md new file mode 100644 index 000000000..f2bc9ac31 --- /dev/null +++ b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.2.3.md @@ -0,0 +1,12 @@ +--- +title: Marten 0.2.3 release notes +pagination_prev: null +pagination_next: null +--- + +_March 23, 2023._ + +## Bug fixes + +* Fix possible incorrect migration dependency generation when two separate apps have migrations generated at the same time +* Fix possible incorrect circular dependency error raised when verifying the acyclic property of the migrations graph diff --git a/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.2.4.md b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.2.4.md new file mode 100644 index 000000000..b610542ff --- /dev/null +++ b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.2.4.md @@ -0,0 +1,12 @@ +--- +title: Marten 0.2.4 release notes +pagination_prev: null +pagination_next: null +--- + +_May 15, 2023._ + +## Bug fixes + +* Fix possible `KeyError` exception raised when the [use_x_forwarded_proto](../../development/reference/settings#use_x_forwarded_proto) setting is set to `true`. +* Fix overly strict route name rules. diff --git a/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.2.md b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.2.md new file mode 100644 index 000000000..bb2a5ab9d --- /dev/null +++ b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.2.md @@ -0,0 +1,133 @@ +--- +title: Marten 0.2.0 release notes +pagination_prev: null +pagination_next: null +--- + +_February 11, 2023._ + +## Requirements and compatibility + +Crystal 1.6 and 1.7. + +## New features + +### Authentication + +The framework now provides the ability to generate projects with a [built-in authentication system](../../authentication) that handles basic user management needs: signing in/out users, resetting passwords, etc. This can be achieved through the use of the `--with-auth` option of the [`new`](../../development/reference/management-commands#new) management command: + +```bash +marten new project myblog --with-auth +``` + +The `--with-auth` option ensures that an `auth` [application](../../development/applications) is created for the generated project under the `src/auth` folder. This application is part of the created project and provides the necessary [models](../../models-and-databases), [handlers](../../handlers-and-http), [schemas](../../schemas), [emails](../../emailing), and [templates](../../templates) allowing authenticating users with email addresses and passwords, while also supporting standard password reset flows. All these abstractions provide a "default" authentication implementation that can be then updated on a per-project basis to better accommodate authentication-related requirements. + +### Email sending + +Marten now lets you define [emails](../../emailing) that you can fully customize (properties, header values, etc) and whose bodies (HTML and/or text) are rendered by leveraging [templates](../../templates). For example, here is how to define a simple email and how to deliver it: + +```crystal +class WelcomeEmail < Marten::Email + to @user.email + subject "Hello!" + template_name "emails/welcome_email.html" + + def initialize(@user : User) + end +end + +email = WelcomeEmail.new(user) +email.deliver +``` + +Emails are delivered by leveraging an [emailing backend mechanism](../../emailing/introduction#emailing-backends). Emailing backends implement _how_ emails are actually sent and delivered. Presently, Marten supports one built-in [development emailing backend](../../emailing/reference/backends#development-backend), and a set of other [third-party backends](../../emailing/reference/backends#other-backends) that you can install depending on your email sending requirements. + +Please refer to the [Emailing section](../../emailing) to learn more about this new feature. + +### Raw SQL capabilities + +Query sets now provide the ability to perform raw queries that are mapped to actual model instances. This is interesting if the capabilities provided by query sets are not sufficient for the task at hand and you need to write custom SQL queries. + +For example: + +```crystal +Article.raw("SELECT * FROM articles WHERE title = ?", "Hello World!").each do |article| + # Do something with `article` record +end +``` + +Please refer to [Raw SQL](../../models-and-databases/raw-sql) to learn more about this capability. + +### Email field for models and schemas + +It is now possible to define `email` fields in [models](../../models-and-databases/reference/fields#email) and [schemas](../../schemas/reference/fields#email). These allow you to easily persist valid email addresses in your models but also to expect valid email addresses in data validated through the use of schemas. + +For example: + +```crystal +class User < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :email, :email, unique: true +end +``` + +### Transaction callbacks + +Models now support the definition of transaction callbacks by using the [`#after_commit`](../../models-and-databases/callbacks#aftercommit) and [`#after_rollback`](../../models-and-databases/callbacks#afterrollback) macros. + +For example: + +```crystal +class User < Marten::Model + field :id, :big_int, primary_key: true, auto: true + field :username, :string, max_size: 64, unique: true + + after_commit :do_something, on: :update + + private def do_something + # Do something! + end +end +``` + +Please refer to [Callbacks](../../models-and-databases/callbacks) to learn more about this capability. + +### Minor features + +#### Models and databases + +* Support for [DB connection pool parameters](https://crystal-lang.org/reference/database/connection_pool.html) was added. See the [database settings reference](../../development/reference/settings#database-settings) for more details about the supported parameters +* Model fields now contribute `#?` methods to model classes in order to easily identify if a field has a value or not. Note that this capability is also enabled for the relationship methods provided by the [`many_to_one`](../../models-and-databases/reference/fields#many_to_one) and [`one_to_one`](../../models-and-databases/reference/fields#one_to_one) fields +* It is now possible to leverage a [`#with_timestamp_fields`](pathname:///api/0.3/Marten/DB/Model/Table.html#with_timestamp_fields-macro) macro to automatically create `created_at` / `updated_at` timestamp fields in models. The `created_at` field is populated with the current time when new records are created while the `updated_at` field is refreshed with the current time whenever records are updated. See [Timestamps](../../models-and-databases/introduction#timestamps) to learn more about this capability +* It is now possible to easily retrieve specific column values without loading entire record objects by leveraging the [`#pluck`](../../models-and-databases/reference/query-set#pluck) and [`#pick`](../../models-and-databases/reference/query-set#pick) query set methods + +#### Handlers and HTTP + +* `string` can now be used as an alias for `str` when defining [routing parameters](../../handlers-and-http/routing#specifying-route-parameters) + +#### Templates + +* Support for index lookups was added. This means that it is now possible to retrieve specific index values from indexable objects in templates using the `{{ var. }}` syntax (for example: `{{ var.0 }}`) +* A [`linebreaks`](../../templates/reference/filters#linebreaks) template filter was introduced to allow replacing all newlines with HTML line breaks (`
`) in strings easily + +#### Schemas + +* Schemas can now be initialized from handler routing parameters (which means that they can be initialized from the hashes that are returned by the [`Marten::Handlers::Base#params`](pathname:///api/0.3/Marten/Handlers/Base.html#params%3AHash(String%2CInt16|Int32|Int64|Int8|String|UInt16|UInt32|UInt64|UInt8|UUID)-instance-method) method) + +#### Management commands + +* The [`serve`](../../development/reference/management-commands#serve) management command now supports the ability to override the host and port used by the development server + +#### Testing + +* A [testing client](../../development/testing#using-the-test-client) was introduced in order to allow developers to easily test handlers (and the various routes of a project) by issuing requests and by introspecting the returned responses. + +## Backward incompatible changes + +### Templates + +* The `loop.first` and `loop.last` for loop variables were respectively renamed `loop.first?` and `loop.last?`. See the [template tag reference](../../templates/reference/tags#for) to learn more about the `for` template tag + +### Management commands + +* The [`new`](../../development/reference/management-commands#new) management command's optional directory argument was replaced by a dedicated `-d DIR, --dir=DIR` option diff --git a/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.3.md b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.3.md new file mode 100644 index 000000000..310ac1002 --- /dev/null +++ b/docs/versioned_docs/version-0.3/the-marten-project/release-notes/0.3.md @@ -0,0 +1,153 @@ +--- +title: Marten 0.3.0 release notes +pagination_prev: null +pagination_next: null +--- + +_June 19, 2023._ + +## Requirements and compatibility + +Crystal 1.6, 1.7, and 1.8. + +## New features + +### Support for streaming responses + +It is now possible to generate streaming responses from iterators of strings easily by leveraging the [`Marten::HTTP::Response::Streaming`](pathname:///api/0.3/Marten/HTTP/Response/Streaming.html) class or the [`#respond`](pathname:///api/0.3/Marten/Handlers/Base.html#respond(streamed_content%3AIterator(String)%2Ccontent_type%3DHTTP%3A%3AResponse%3A%3ADEFAULT_CONTENT_TYPE%2Cstatus%3D200)-instance-method) helper method. This can be beneficial if you intend to generate lengthy responses or responses that consume excessive memory (a classic example of this is the generation of large CSV files). + +Please refer to [Streaming responses](../../handlers-and-http/introduction#streaming-responses) to learn more about this new capability. + +### Caching + +Marten now lets you interact with a global cache store that allows interacting with an underlying cache system and performing basic operations such as fetching cached entries, writing new entries, etc. By using caching, you can save the result of expensive operations so that you don't have to perform them for every request. + +The global cache can be accessed by leveraging the [`Marten#cache`](pathname:///api/0.3/Marten.html#cache%3ACache%3A%3AStore%3A%3ABase-class-method) method. Here are a few examples on how to perform some basic caching operations: + +```crystal +# Fetching an entry from the cache. +Marten.cache.fetch("mykey", expires_in: 4.hours) do + "myvalue" +end + +# Reading from the cache. +Marten.cache.read("unknown") # => nil +Marten.cache.read("mykey") # => "myvalue" +Marten.cache.exists?("mykey") => true + +# Writing to the cache. +Marten.cache.write("foo", "bar", expires_in: 10.minutes) => true +``` + +Marten's caching leverages a [cache store mechanism](../../caching/introduction#configuration-and-cache-stores). By default, Marten uses an in-memory cache (instance of [`Marten::Cache::Store::Memory`](pathname:///api/0.3/Marten/Cache/Store/Memory.html)) and other [third-party stores](../../caching/reference/stores#other-stores) can be installed depending on your caching requirements (eg. Memcached, Redis). + +Marten's new caching capabilities are not only limited to its standard cache functionality. They can also be effectively utilized via the newly introduced [template fragment caching](../../caching/introduction#template-fragment-caching) feature, made possible by the [`cache`](../../templates/reference/tags#cache) template tag. With this feature, specific parts of your [templates](../../templates) can now be cached with ease. + +Please refer to the [Caching](../../caching) to learn more about these new capabilities. + +### JSON field for models and schemas + +Marten now provides the ability to define `json` fields in [models](../../models-and-databases/reference/fields#json) and [schemas](../../schemas/reference/fields#json). These fields allow you to easily persist and interact with valid JSON structures that are exposed as [`JSON::Any`](https://crystal-lang.org/api/JSON/Any.html) objects by default. + +For example: + +```crystal +class MyModel < Marten::Model + # Other fields... + field :metadata, :json +end + +MyModel.last!.metadata # => JSON::Any object +``` + +Additionally, it is also possible to specify that JSON values must be deserialized using a class that makes use of [`JSON::Serializable`](https://crystal-lang.org/api/JSON/Serializable.html). This can be done by leveraging the `serializable` option in both [model fields](../../models-and-databases/reference/fields#json) and [schema fields](../../schemas/reference/fields#serializable). + +For example: + +```crystal +class MySerializable + include JSON::Serializable + + property a : Int32 | Nil + property b : String | Nil +end + +class MyModel < Marten::Model + # Other fields... + field :metadata, :json, serializable: MySerializable +end + +MyModel.last!.metadata # => MySerializable object +``` + +### Duration field for models and schemas + +It is now possible to define `duration` fields in [models](../../models-and-databases/reference/fields#duration) and [schemas](../../schemas/reference/fields#duration). These allow you to easily persist valid durations (that map to [`Time::Span`](https://crystal-lang.org/api/Time/Span.html) objects in Crystal) in your models but also to expect valid durations in data validated through the use of schemas. + +For example: + +```crystal +class Recipe < Marten::Model + field :id, :big_int, primary_key: true, auto: true + # Other fields... + field :fridge_time, :duration, blank: true, null: true +end +``` + +### Minor features + +#### Models and databases + +* New [`#get_or_create`](../../models-and-databases/reference/query-set#get_or_create) / [`#get_or_create!`](../../models-and-databases/reference/query-set#get_or_create-1) methods were added to query sets in order to allow easily retrieving a model record matching a given set of filters or creating a new one if no record is found. +* [`string`](../../models-and-databases/reference/fields#string) fields now support a `min_size` option allowing to validate the minimum size of persisted string field values. +* A new [`#includes?`](../../models-and-databases/reference/query-set#includes) method was added to query sets in order easily perform membership checks without loading the entire list of records targeted by a given query set. +* Alternative [`#exists?`](../../models-and-databases/reference/query-set#exists) methods were added to query sets in order to allow specifying additional filters to use as part of existence checks. +* An [`#any?`](pathname:///api/0.3/Marten/DB/Query/Set.html#any%3F-instance-method) method was added to query sets in order to short-circuit the default implementation of [`Enumerable#any?`](https://crystal-lang.org/api/Enumerable.html#any%3F%3ABool-instance-method) and to avoid loading the full list of records in memory (when called without arguments). This overridden method is technically an alias of [`#exists?`](../../models-and-databases/reference/query-set#exists). +* Marten migrations are now optimized to prevent possible issues with circular dependencies within added or deleted tables +* It is now possible to define arbitrary database options by using the new [`db.options`](../../development/reference/settings#options) database setting. +* It is now possible to define [`many_to_one`](../../models-and-databases/reference/fields#many_to_one) and [`one_to_one`](../../models-and-databases/reference/fields#one_to_one) fields that target models with non-integer primary key fields (such as UUID fields for example). + +#### Handlers and HTTP + +* The [`Marten::Handlers::RecordList`](../../handlers-and-http/reference/generic-handlers#listing-records) generic record now provides the ability to specify a custom [query set](../../models-and-databases/queries) instead of a model class. This can be achieved through the use of the [`#queryset`](pathname:///api/0.3/Marten/Handlers/RecordListing.html#queryset(queryset)-macro) macro. +* A new [`Marten::Middleware::AssetServing`](../../handlers-and-http/reference/middlewares#asset-serving-middleware) middleware was introduced to make it easy to serve collected assets in situations where it is not possible to easily configure a web server (such as [Nginx](https://nginx.org)) or a third-party service (like Amazon's S3 or GCS) to serve assets directly. +* A new [`Marten::Middleware::SSLRedirect`](../../handlers-and-http/reference/middlewares#ssl-redirect-middleware) middleware was introduced to allow redirecting non-HTTPS requests to HTTPS easily. +* A new [`Marten::Middleware::ContentSecurityPolicy`](../../handlers-and-http/reference/middlewares#content-security-policy-middleware) middleware was introduced to ensure the presence of the Content-Security-Policy header in the response's headers. Please refer to [Content Security Policy](../../security/content-security-policy) to learn more about the Content-Security-Policy header and how to configure it. +* The [`Marten::Middleware::I18n`](../../handlers-and-http/reference/middlewares#i18n-middleware) middleware can now automatically determine the current locale based on the value of a cookie whose name can be configured with the [`i18n.locale_cookie_name`](../../development/reference/settings#locale_cookie_name) setting. +* The [`Marten::Middleware::I18n`](../../handlers-and-http/reference/middlewares#i18n-middleware) middleware now automatically sets the Content-Language header based on the activated locale. + +#### Templates + +* A [`join`](../../templates/reference/filters#join) template filter was introduced to allow converting enumerable template values into a string separated by a separator value. +* A [`split`](../../templates/reference/filters#split) template filter was introduced to allow converting a string into an array of elements. + +#### Schemas + +* Type-safe getter methods (ie. `#`, `#!`, and `#?`) are now automatically generated for schema fields. Please refer to [Accessing validated data](../../schemas/introduction#accessing-validated-data) in the schemas documentation to read more about these methods and how/why to use them. + +#### Development + +* [`Marten::HTTP::Errors::SuspiciousOperation`](pathname:///api/0.3/Marten/HTTP/Errors/SuspiciousOperation.html) exceptions are now showcased using the debug internal error page handler to make it easier to diagnose errors such as unexpected host errors (which result from a missing host value in the [`allowed_hosts`](../../development/reference/settings#allowedhosts) setting). +* [`Marten#setup`](pathname:///api/0.3/Marten.html#setup-class-method) now raises.[`Marten::Conf::Errors::InvalidConfiguration`](pathname:///api/0.3/Marten/Conf/Errors/InvalidConfiguration.html) exceptions when a configured database involves a backend that is not installed (eg. a MySQL database configured without `crystal-lang/crystal-mysql` installed and required). +* The [`new`](../../development/reference/management-commands#new) management command now automatically creates a [`.editorconfig`](https://editorconfig.org) file for new projects. +* A new [`root_path`](../../development/reference/settings#root_path) setting was introduced to make it possible to configure the actual location of the project sources in your system. This is especially useful when deploying projects that have been compiled in a different location from their final destination, which can happen on platforms like Heroku. By setting the root path, you can ensure that your application can find all necessary project sources, as well as other files like locales, assets, and templates. +* A new `--plan` option was added to the [`migrate`](../../development/reference/management-commands#migrate) management command in order to provide a comprehensive overview of the operations that will be performed by the applied or unapplied migrations. +* An interactive mode was added to the [`new`](../../development/reference/management-commands#new) management command: if the `type` and `name` arguments are not provided, the command now prompts the user for inputting the structure type, the app or project name, and whether the auth app should be generated. +* It is now possible to specify command aliases when defining management commands by leveraging the [`#command_aliases`](pathname:///api/0.3/Marten/CLI/Manage/Command/Base.html#command_aliases(*aliases%3AString|Symbol)-class-method) helper method. + +#### Security + +* The ability to fully configure and customize the Content-Security-Policy header was added to the framework. Please refer to [Content Security Policy](../../security/content-security-policy) to learn more about the Content-Security-Policy header and how to configure it in Marten projects. + +#### Deployment + +* A new guide was added in order to document [how to deploy on Heroku](../../deployment/how-to/deploy-to-heroku). +* A new guide was added in order to document [how to deploy on Fly.io](../../deployment/how-to/deploy-to-fly-io). + +## Backward incompatible changes + +### Handlers and HTTP + +* [Custom route parameter](../../handlers-and-http/how-to/create-custom-route-parameters) must now implement a [`#regex`](pathname:///api/0.3/Marten/Routing/Parameter/Base.html#regex%3ARegex-instance-method) method and can no longer rely on a `#regex` macro to generate such method. +* The generic handlers that used to require the use of a `#model` class method now leverage a dedicated macro instead. This is to make handlers that inherit from generic handler classes more type-safe when it comes to manipulating model records. +* The generic handlers that used to require the use of a `#schema` class method now leverage a dedicated macro instead. This is to make handlers that inherit from generic handler classes more type-safe when it comes to manipulating schema instances. diff --git a/docs/versioned_sidebars/version-0.3-sidebars.json b/docs/versioned_sidebars/version-0.3-sidebars.json new file mode 100644 index 000000000..5c6f6629b --- /dev/null +++ b/docs/versioned_sidebars/version-0.3-sidebars.json @@ -0,0 +1,348 @@ +{ + "sidebar": [ + "prologue", + { + "type": "category", + "label": "Getting Started", + "link": { + "type": "doc", + "id": "getting-started" + }, + "items": [ + "getting-started/installation", + "getting-started/tutorial" + ] + }, + { + "type": "category", + "label": "Models and databases", + "link": { + "type": "doc", + "id": "models-and-databases" + }, + "items": [ + "models-and-databases/introduction", + "models-and-databases/queries", + "models-and-databases/validations", + "models-and-databases/callbacks", + "models-and-databases/migrations", + "models-and-databases/transactions", + "models-and-databases/raw-sql", + "models-and-databases/multiple-databases", + { + "type": "category", + "label": "How-To's", + "items": [ + "models-and-databases/how-to/create-custom-model-fields" + ] + }, + { + "type": "category", + "label": "Reference", + "items": [ + "models-and-databases/reference/fields", + "models-and-databases/reference/table-options", + "models-and-databases/reference/query-set", + "models-and-databases/reference/migration-operations" + ] + } + ] + }, + { + "type": "category", + "label": "Handlers and HTTP", + "link": { + "type": "doc", + "id": "handlers-and-http" + }, + "items": [ + "handlers-and-http/introduction", + "handlers-and-http/routing", + "handlers-and-http/generic-handlers", + "handlers-and-http/error-handlers", + "handlers-and-http/sessions", + "handlers-and-http/cookies", + "handlers-and-http/middlewares", + { + "type": "category", + "label": "How-To's", + "items": [ + "handlers-and-http/how-to/create-custom-route-parameters" + ] + }, + { + "type": "category", + "label": "Reference", + "items": [ + "handlers-and-http/reference/generic-handlers", + "handlers-and-http/reference/middlewares" + ] + } + ] + }, + { + "type": "category", + "label": "Templates", + "link": { + "type": "doc", + "id": "templates" + }, + "items": [ + "templates/introduction", + { + "type": "category", + "label": "How-To's", + "items": [ + "templates/how-to/create-custom-filters", + "templates/how-to/create-custom-tags", + "templates/how-to/create-custom-context-producers" + ] + }, + { + "type": "category", + "label": "Reference", + "items": [ + "templates/reference/filters", + "templates/reference/tags", + "templates/reference/context-producers" + ] + } + ] + }, + { + "type": "category", + "label": "Schemas", + "link": { + "type": "doc", + "id": "schemas" + }, + "items": [ + "schemas/introduction", + "schemas/validations", + { + "type": "category", + "label": "How-To's", + "items": [ + "schemas/how-to/create-custom-schema-fields" + ] + }, + { + "type": "category", + "label": "Reference", + "items": [ + "schemas/reference/fields" + ] + } + ] + }, + { + "type": "category", + "label": "Assets", + "link": { + "type": "doc", + "id": "assets" + }, + "items": [ + "assets/introduction" + ] + }, + { + "type": "category", + "label": "Files", + "link": { + "type": "doc", + "id": "files" + }, + "items": [ + "files/uploading-files", + "files/managing-files", + { + "type": "category", + "label": "How-To's", + "items": [ + "files/how-to/create-custom-file-storages" + ] + } + ] + }, + { + "type": "category", + "label": "Development", + "link": { + "type": "doc", + "id": "development" + }, + "items": [ + "development/settings", + "development/applications", + "development/management-commands", + "development/testing", + { + "type": "category", + "label": "How-To's", + "items": [ + "development/how-to/create-custom-commands" + ] + }, + { + "type": "category", + "label": "Reference", + "items": [ + "development/reference/settings", + "development/reference/management-commands" + ] + } + ] + }, + { + "type": "category", + "label": "Security", + "link": { + "type": "doc", + "id": "security" + }, + "items": [ + "security/introduction", + "security/csrf", + "security/clickjacking", + "security/content-security-policy" + ] + }, + { + "type": "category", + "label": "Internationalization", + "link": { + "type": "doc", + "id": "i18n" + }, + "items": [ + "i18n/introduction" + ] + }, + { + "type": "category", + "label": "Emailing", + "link": { + "type": "doc", + "id": "emailing" + }, + "items": [ + "emailing/introduction", + { + "type": "category", + "label": "How-To's", + "items": [ + "emailing/how-to/create-custom-emailing-backends" + ] + }, + { + "type": "category", + "label": "Reference", + "items": [ + "emailing/reference/backends" + ] + } + ] + }, + { + "type": "category", + "label": "Authentication", + "link": { + "type": "doc", + "id": "authentication" + }, + "items": [ + "authentication/introduction", + { + "type": "category", + "label": "Reference", + "items": [ + "authentication/reference/generated-files" + ] + } + ] + }, + { + "type": "category", + "label": "Caching", + "link": { + "type": "doc", + "id": "caching" + }, + "items": [ + "caching/introduction", + { + "type": "category", + "label": "How-To's", + "items": [ + "caching/how-to/create-custom-cache-stores" + ] + }, + { + "type": "category", + "label": "Reference", + "items": [ + "caching/reference/stores" + ] + } + ] + }, + { + "type": "category", + "label": "Deployment", + "link": { + "type": "doc", + "id": "deployment" + }, + "items": [ + "deployment/introduction", + { + "type": "category", + "label": "How-To's", + "items": [ + "deployment/how-to/deploy-to-an-ubuntu-server", + "deployment/how-to/deploy-to-heroku", + "deployment/how-to/deploy-to-fly-io" + ] + } + ] + }, + { + "type": "category", + "label": "The Marten project", + "link": { + "type": "doc", + "id": "the-marten-project" + }, + "items": [ + "the-marten-project/contributing", + "the-marten-project/design-philosophies", + "the-marten-project/acknowledgments", + { + "type": "category", + "label": "Release notes", + "link": { + "type": "doc", + "id": "the-marten-project/release-notes" + }, + "collapsible": false, + "className": "release-notes", + "items": [ + "the-marten-project/release-notes/0.1", + "the-marten-project/release-notes/0.1.1", + "the-marten-project/release-notes/0.1.2", + "the-marten-project/release-notes/0.1.3", + "the-marten-project/release-notes/0.1.4", + "the-marten-project/release-notes/0.1.5", + "the-marten-project/release-notes/0.2", + "the-marten-project/release-notes/0.2.1", + "the-marten-project/release-notes/0.2.2", + "the-marten-project/release-notes/0.2.3", + "the-marten-project/release-notes/0.2.4", + "the-marten-project/release-notes/0.3" + ] + } + ] + } + ] +} diff --git a/docs/versions.json b/docs/versions.json index 7732af685..840caf92f 100644 --- a/docs/versions.json +++ b/docs/versions.json @@ -1,4 +1,5 @@ [ + "0.3", "0.2", "0.1" ] diff --git a/shard.yml b/shard.yml index 6bf3c8363..dc1db3e69 100644 --- a/shard.yml +++ b/shard.yml @@ -1,5 +1,5 @@ name: marten -version: 0.2.4 +version: 0.3.0 authors: - Morgan Aubert diff --git a/src/marten.cr b/src/marten.cr index 01b72d6ad..63bb9ca6f 100644 --- a/src/marten.cr +++ b/src/marten.cr @@ -40,7 +40,7 @@ require "./marten/server/**" require "./marten/template/**" module Marten - VERSION = "0.2.4" + VERSION = "0.3.0" Log = ::Log.for("marten")