Sync open source content 🐝 (from 10787ea1995f682e6036a30eee53ceeeb0ce3217)

Author: beezythebot

api-design/caching.mdx

@@ -7,15 +7,15 @@ import { Callout } from "@/mdx/components";
 
 # Caching API Responses
 
-API caching can save servers some serious work, cut down on costs, and even help reduce the carbon impact of an API. However, it is often considered an optimization rather than what it truly is: an integral part of API design. 
+API caching can save servers some serious work, cut down on costs, and even help reduce the carbon impact of an API. However, it is often considered an optimization rather than what it truly is: an integral part of API design.
 
 A fundamental part of REST is APIs declaring the "cacheability" of resources. When working with HTTP there are many amazing caching options available through HTTP Caching; a series of standards that power how the entire internet functions. This can be used to design more useful APIs, as well as being faster, cheaper, and more sustainable.
 
 ## What is HTTP caching?
 
 HTTP caching tells API clients (like browsers, mobile apps, or other backend systems) if they need to ask for the same data over and over again, or if they can use data they already have. This is done with HTTP headers on responses that tell the client how long they can "hold onto" that response, or how to check if it's still valid.
 
-This works very differently from server-side caching tools like Redis or Memcached, which cache data on the server. 
+This works very differently from server-side caching tools like Redis or Memcached, which cache data on the server.
 
 HTTP caching happens on client-side or on intermediary proxies like Content Delivery Networks (CDNs), acting as a proxy between the client and the server and storing responses for reuse whenever possible.
 
@@ -88,32 +88,32 @@ All of this is done without the client needing to know anything about the data,
 Let's add these headers to a basic Express.js API to see how it might look on the server-side.
 
 ```js
-const express = require('express');
+const express = require("express");
 const app = express();
 
-app.get('/api/resource', (req, res) => {
-    const data = { message: "Hello, world!" }; // Simulated data
-    const eTag = `"${Buffer.from(JSON.stringify(data)).toString('base64')}"`;
-
-    if (req.headers['if-none-match'] === eTag) {
-        // Client has the latest version
-        res.status(304).end();
-    } else {
-        // Serve the resource with cache headers
-        res.set({
-            'Cache-Control': 'max-age=3600', // Cache for 1 hour
-            'ETag': eTag
-        });
-        res.json(data);
-    }
+app.get("/api/resource", (req, res) => {
+  const data = { message: "Hello, world!" }; // Simulated data
+  const eTag = `"${Buffer.from(JSON.stringify(data)).toString("base64")}"`;
+
+  if (req.headers["if-none-match"] === eTag) {
+    // Client has the latest version
+    res.status(304).end();
+  } else {
+    // Serve the resource with cache headers
+    res.set({
+      "Cache-Control": "max-age=3600", // Cache for 1 hour
+      ETag: eTag,
+    });
+    res.json(data);
+  }
 });
 
-app.listen(3000, () => console.log('API running on http://localhost:3000'));
+app.listen(3000, () => console.log("API running on http://localhost:3000"));
 ```
 
 The ETag is generated by hashing the data, then the server checks if the client has the latest version. If it does, it sends a `304 Not Modified` response, otherwise it sends the data with the `ETag` and `Cache-Control` headers.
 
-In a real codebase, would be doing something like fetching from a datasource, or computing something that takes a while, so waiting for all of that to happen just to make an ETag is not ideal. Yes, it avoids turning that data in JSON and sending it over the wire, but if the API is going to ignore it and send an `304 Not Modified` header with no response, the data was loaded and hashed for no reason. 
+In a real codebase, would be doing something like fetching from a datasource, or computing something that takes a while, so waiting for all of that to happen just to make an ETag is not ideal. Yes, it avoids turning that data in JSON and sending it over the wire, but if the API is going to ignore it and send an `304 Not Modified` header with no response, the data was loaded and hashed for no reason.
 
 Instead, an ETag can be made from metadata, like the last updated timestamp of a database record.
 
@@ -156,7 +156,10 @@ Using `Cache-Control` headers its possible to specify whether the response can b
 - `no-store` — The response can't be cached at all.
 
 <Callout title="Note" type="info">
-  When a response contains an `Authorization` header, it's automatically marked as `private` to prevent sensitive data from being cached by shared caches. This is another reason to use standard auth headers instead of using custom headers like `X-API-Key`.
+  When a response contains an `Authorization` header, it's automatically marked
+  as `private` to prevent sensitive data from being cached by shared caches.
+  This is another reason to use standard auth headers instead of using custom
+  headers like `X-API-Key`.
 </Callout>
 
 ## Which resources should be cached?
@@ -188,17 +191,17 @@ GET /invoices/645E79D9E14
   "id": "645E79D9E14",
   "invoiceNumber": "INV-2024-001",
   "customer": "Acme Corporation",
-  "amountDue": 500.00,
-  "amountPaid": 250.00,
+  "amountDue": 500.0,
+  "amountPaid": 250.0,
   "dateDue": "2024-08-15",
   "dateIssued": "2024-08-01",
   "datePaid": "2024-08-10",
   "items": [
     {
       "description": "Consulting Services",
       "quantity": 10,
-      "unitPrice": 50.00,
-      "total": 500.00
+      "unitPrice": 50.0,
+      "total": 500.0
     }
   ],
   "customer": {
@@ -213,15 +216,15 @@ GET /invoices/645E79D9E14
   "payments": [
     {
       "date": "2024-08-10",
-      "amount": 250.00,
+      "amount": 250.0,
       "method": "Credit Card",
       "reference": "CC-1234"
     }
   ]
 }
 ```
 
-This is a very common pattern, but it's not very cacheable. If the invoice is updated, the whole invoice is updated, and the whole invoice needs to be refreshed. If the customer is updated, the whole invoice is updated, and the whole invoice needs to be refreshed. If the payments are updated, the whole invoice is updated, and the whole invoice needs to be refreshed. 
+This is a very common pattern, but it's not very cacheable. If the invoice is updated, the whole invoice is updated, and the whole invoice needs to be refreshed. If the customer is updated, the whole invoice is updated, and the whole invoice needs to be refreshed. If the payments are updated, the whole invoice is updated, and the whole invoice needs to be refreshed.
 
 We can increase the cachability of most of this information by breaking it down into smaller resources:
 
@@ -234,15 +237,15 @@ GET /invoices/645E79D9E14
   "id": "645E79D9E14",
   "invoiceNumber": "INV-2024-001",
   "customer": "Acme Corporation",
-  "amountDue": 500.00,
+  "amountDue": 500.0,
   "dateDue": "2024-08-15",
   "dateIssued": "2024-08-01",
   "items": [
     {
       "description": "Consulting Services",
       "quantity": 10,
-      "unitPrice": 50.00,
-      "total": 500.00
+      "unitPrice": 50.0,
+      "total": 500.0
     }
   ],
   "links": {
@@ -257,15 +260,15 @@ Instead of mixing in payment information with the invoice, this example moves th
 
 The customer data is also moved out of the invoice resource, because the `/customers/acme-corporation` resource already exists and reusing it avoids code duplication and maintenance burden. Considering the user flow of the application, the resource is likely already in the browser/client cache, which reduces load times for the invoice.
 
-This API structure works regardless of what the data structure looks like. Perhaps all of the payment data are in an `invoices` SQL table, but still have `/invoices` and `/invoices/{id}/payments` endpoints. Over time as common extra functionality like partial payments is requested, these endpoints can remain the same, but the underlying database structure can be migrated to move payment-specific fields over to a `payments` database table. 
+This API structure works regardless of what the data structure looks like. Perhaps all of the payment data are in an `invoices` SQL table, but still have `/invoices` and `/invoices/{id}/payments` endpoints. Over time as common extra functionality like partial payments is requested, these endpoints can remain the same, but the underlying database structure can be migrated to move payment-specific fields over to a `payments` database table.
 
-Many would argue this is a better separation of concerns, it's easier to control permissions for who is allowed to see invoices and/or payments, and the API has drastically improved cachability by splitting out frequently changing information from rarely changing information. 
+Many would argue this is a better separation of concerns, it's easier to control permissions for who is allowed to see invoices and/or payments, and the API has drastically improved cachability by splitting out frequently changing information from rarely changing information.
 
 ### Avoid mixing public and private data
 
-Breaking things down into smaller, more manageable resources can separate frequently changing information from more stable data, but there are other design issues that can effect cachability: mixing public and private data. 
+Breaking things down into smaller, more manageable resources can separate frequently changing information from more stable data, but there are other design issues that can effect cachability: mixing public and private data.
 
-Take the example of a train travel booking API. There could be a Booking resource, specific to a single user with private data nobody else should see. 
+Take the example of a train travel booking API. There could be a Booking resource, specific to a single user with private data nobody else should see.
 
 ```http
 GET /bookings/1234
@@ -310,15 +313,15 @@ There is no downside to caching this data, because it is the same for everyone.
 
 ## Content Delivery Networks (CDNs)
 
-HTTP caching works well when clients use it, and many do automatically, like web browsers or systems with caching middleware. But it becomes even more powerful when combined with tools like [Fastly](https://www.fastly.com/) or [Varnish](https://www.varnish-software.com/products/varnish-cache/). 
+HTTP caching works well when clients use it, and many do automatically, like web browsers or systems with caching middleware. But it becomes even more powerful when combined with tools like [Fastly](https://www.fastly.com/) or [Varnish](https://www.varnish-software.com/products/varnish-cache/).
 
 These tools sit between the server and the client, acting like intelligent gatekeepers:
 
 ![A sequence diagram showing a Client, Cache Proxy, and Server. A web request travels from client to proxy, then is sent on to the server, showing a "cache miss". The response then travels back from the server to the cache proxy, and then is sent to the client](./assets/httpcachemiss.png)
 
 ![A sequence diagram showing a Client, Cache Proxy, and Server. A web request travels from client to proxy, but does not go to the server, showing show a "cache hit". The response is served from the cache proxy to the client without involving the server](./assets/httpcachehit.png)
 
-Client-caching like this is certainly useful, but the real power of caching comes when API web traffic is routed through a caching proxy. Using hosted solutions like Fastly or AWS CloudFront, this could be a case of changing DNS settings. For self-hosted options like Varnish, instead of pointing DNS settings to a hosted solution somebody will need to spin up a server to act as the cache proxy. 
+Client-caching like this is certainly useful, but the real power of caching comes when API web traffic is routed through a caching proxy. Using hosted solutions like Fastly or AWS CloudFront, this could be a case of changing DNS settings. For self-hosted options like Varnish, instead of pointing DNS settings to a hosted solution somebody will need to spin up a server to act as the cache proxy.
 
 Many API gateway tools like Tyk and Zuplo have caching built in, so this functionality may already be available in the ecosystem and just need enabling.
 

api-design/pagination.mdx

@@ -3,7 +3,7 @@ title: "Pagination Best Practices in REST API Design"
 description: "Implement efficient pagination in your API to handle large datasets, improve performance, and provide a better experience for API consumers."
 ---
 
-import { Callout } from '@/mdx/components';
+import { Callout } from "@/mdx/components";
 
 # Paginating API responses
 
@@ -17,7 +17,7 @@ stuck into doing things the right way early on.
 At first it's easy to imagine that collections only have a few hundred records.
 That not be too taxing for the server to fetch from the database, turn into
 JSON, and send back to the client, but as soon as the collection is getting into
-thousands of records things start to fall apart in wild and unexpected ways. 
+thousands of records things start to fall apart in wild and unexpected ways.
 
 For example, a coworking company that expected to mostly host startups of 10-50
 people, but then Facebook and Amazon rock up with ~10,000 employees each, and
@@ -83,7 +83,7 @@ The best way to help the client is to give them links, which at first seems
 confusing but it's just
 [HATEOAS](https://apisyouwonthate.com/blog/rest-and-richardson-maturity-model/)
 (Hypermedia As The Engine Of Application State), also known as Hypermedia
-Controls. 
+Controls.
 
 It's a fancy way of saying "give them links for things they can do
 next" and in the context of pagination that means "give them links to the next
@@ -110,14 +110,14 @@ page, the previous page, the first page, and the last page."
 ```
 
 Whenever there is a `next` link, an API consumer can show a `next` button, or
-start loading the next page of data to allow for auto-scrolling. 
+start loading the next page of data to allow for auto-scrolling.
 
 If the `next` response returns data, it will give a 200 OK response and they can
-show the data. 
+show the data.
 
 If there is no data then it will still be a 200 OK but there will be an empty
 array, showing that everything was fine, but there is no data on that page right
-now. 
+now.
 
 **Ease of Use**
 
@@ -134,10 +134,10 @@ now.
 **Consistency**
 
 - Con: When a consumer loads the latest 10 records, then a new record is added
-to the database, then a user loads the second page, they'll see one of those
-records twice. This is because there is no such concept as a "page" in the
-database, just saying "grab me 10, now the next 10" does not differentiate which
-records they actually were.
+  to the database, then a user loads the second page, they'll see one of those
+  records twice. This is because there is no such concept as a "page" in the
+  database, just saying "grab me 10, now the next 10" does not differentiate which
+  records they actually were.
 
 ### Offset-Based Pagination
 
@@ -208,7 +208,7 @@ Or with hypermedia controls in the JSON:
 **Consistency**
 
 - Con: The same problems exist for offset pagination as page pagination, if
-more data has been added between the first request and second being made, the same record could show up in both pages.
+  more data has been added between the first request and second being made, the same record could show up in both pages.
 
 **See this in action**
 
@@ -230,7 +230,7 @@ page, this could be a UUID, but it can be more dynamic than that.
 
 APIs like Slack will base64 encode information with a field name and a value,
 even adding sorting logic, all wrapped up in an opaque string. For example,
-`dXNlcjpXMDdRQ1JQQTQ=` would represent `user:W07QCRPA4`. 
+`dXNlcjpXMDdRQ1JQQTQ=` would represent `user:W07QCRPA4`.
 
 Obfuscating the information like this aims to stop API consumers hard-coding
 values for the pagination, which allows for the API to change pagination logic
@@ -292,10 +292,10 @@ Choosing the right pagination strategy depends on the specific use case and
 dataset size.
 
 Offset-based pagination is simple but may suffer from performance issues with
-large datasets. 
+large datasets.
 
 Cursor-based pagination offers better performance and consistency for large
-datasets but come with added complexity. 
+datasets but come with added complexity.
 
 Page-based pagination is user-friendly but shares similar performance concerns
 with offset-based pagination.
@@ -336,5 +336,7 @@ Adding or drastically changing pagination later could be a whole mess of
 backwards compatibility breaks.
 
 <Callout title="Note" type="info">
-  Pagination can be tricky to work with for API clients, but Speakeasy SDKs can help out. Learn about <a href="/docs/runtime/pagination">adding pagination</a> to your Speakeasy SDK.
+  Pagination can be tricky to work with for API clients, but Speakeasy SDKs can
+  help out. Learn about <a href="/docs/runtime/pagination">adding pagination</a>{" "}
+  to your Speakeasy SDK.
 </Callout>