diff --git a/architecture-design/Architectural-boundaries.md b/architecture-design/Architectural-boundaries.md new file mode 100644 index 0000000..be836f7 --- /dev/null +++ b/architecture-design/Architectural-boundaries.md @@ -0,0 +1,328 @@ +# Architectural boundaries + +See: + +- Clean Architecture (book by Robert C. Martin) +- Building Evolutionary Architectures (book by Neal Ford, Rebecca Parsons and Patrick Kua) ([summary slides](https://www.slideshare.net/thekua/building-evolutionary-architectures)) +- [Conway’s Corollary](http://www.ianbicking.org/blog/2015/08/conways-corollary.html) +- [Our Software Dependency Problem](https://research.swtch.com/deps) + +## Architecture is about boundaries + +System's architecture defines shape of the system: + +- How the system is divided into components +- How those components are arranged +- What kinds of boundaries exist between different components +- How components communicate across those boundaries + +Not only technical components but also components within the codebase itself! + +Boundaries separate parts of the system that shouldn’t know too much about each other + +Benefits of this separation: flexibility! + +- Lower coupling: Decouple components so changes in one component don't require changes in other components +- Higher cohesion: Group together things that change at the same rate and for the same reasons (cohesion) + - Manifestation of the Single Responsibility Principle (from the [SOLID principles](./oo-design/SOLID-principles.md)), but this time at the architectural level +- Ability to delay choices until last possible moment +- Example: Separating main business logic from persistence logic + - If the business logic doesn’t have any idea about the database we use (or potentially even the fact that we use a database), we have the flexibility to change the database that our system depends on without having to make any kind of changes to the main business logic + - If we need to make changes to the main business logic that do not influence the kind of data that needs to be persisted, we can make those changes without the persistence code having to know anything about them + - You could potentially build all of the business logic without connecting to a database, simply writing code against some persistence interfaces describing what kind of data you will need to store and retrieve, and thus delay the decision of which kind of database to use until you have a better idea of what you need from your persistence solution +- Example: Separating different functional areas from each other + - Scaling! Can allow different functional parts of the system to be developed by different teams while keeping the required amount of coordination between teams manageable + +Flexibility provided by boundaries is important, especially for maintenance (typically the most risky and expensive part). Often, the first version of a system making it to production is only the start, and most of the work will happen after that. Additional requirements will be added, existing functionality will need to be changed, and so on. Adequate boundaries will provide the necessary flexibility to make this kind of maintenance possible, allowing the system to grow without exponentially increasing the work needed to add or adjust a piece of functionality. + +## Different kinds of boundaries + +### Horizontal versus vertical separation + +#### Horizontal slicing + +- Boundaries between different technical areas (layers) of the system +- Example: a layer for the API, a layer the business logic and a layer for communicating with the database +- Benefit: allow for technological flexibility (for example, relatively easy to switch to other kind of DB) +- Drawback: single functional change is likely to affect multiple layers + +#### Vertical slicing + +- Boundaries between different functional areas of the system +- Example: functionality for managing customers can be separated from functionality for placing orders +- Benefit: Changes within a single functional domain can happen within a single part of the system + - Especially helpful if different parts maintained by different teams! + - Changes within a single functional domain can happen within a single team and coordination with other teams is only required if the communication with other functional domains needs changes as well + - See [microservices](reference-architectures/Microservices.md), where different small teams each maintain one or more microservices that encapsulate a certain functional area across several layers of the technical stack, even down to the database + +#### Conway's Law + +Conway's Law is as follows: + +> organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations + +Such designs actually make sense as well: if changes within a single part of the system can happen within a single team, it’s way easier to plan and execute these changes + +Consequence: f there is a mismatch between the team structure within your organization and the architecture of the application you’re working on, building the application is likely to be a struggle + +You can use Conway’s Law to your advantage by structuring your application (and thus your teams) in such a way that changes to the system are pretty likely to be confined to a single part of the application. In practice, it seems that vertical slicing is typically the best way to do that. + +### Separation mechanisms + +#### Source-level boundaries + +- Lowest-level boundaries +- Use mechanism offered by the programming language (classes, interfaces, packages, modules, …) +- Communication through simple method calls + - Fast, don't have to worry about amount of communication passing boundary +- Not visible at deployment time + - Does not mean they are not important! When set up correctly, they can still help to isolate different parts of the system from each other in order to facilitate independent development by multiple persons or teams +- Only kind of boundary in monolithic systems + +##### Abstraction + +Use your language's abstraction mechanisms to separate clients from implementations + +Example: [Strategy pattern](https://en.wikipedia.org/wiki/Strategy_pattern) + +![Strategy pattern](_img/Architectural-boundaries/strategy-pattern.png) + +##### Encapsulation + +Divide the code into modules where classes outside of a module can only see the classes that that module explicitly exposes + +Examples in Java: + +- Making classes package protected (limitation: Java does not have real notion of nested packages) + +- The [Java Platform Module System](https://en.wikipedia.org/wiki/Java_Platform_Module_System) + +If your language doesn't provide something like this, you may be able to set it up using linting rules or automated tests that analyze dependencies + +##### Data transfer objects + +When passing data between components, the components can be more decoupled if the objects crossing the boundary are simple data transfer objects instead of actual specific classes that are being used on either side + +##### Combining the above + +Example of combining the above: use case boundary as specified by the Clean Architecture + +![Clean Architecture boundary](_img/Architectural-boundaries/clean-architecture-boundary.jpg) + +- Abstraction, specifying interfaces for both client and provider +- Data transfer objects for input and output data +- Ideally, encapsulation is used to prevent clients from calling the implementation(s) of the input boundary interface directly + +Also note that the Dependency Inversion Principle (from the [SOLID principles](./oo-design/SOLID-principles.md)) has been used to make sure all dependencies point from the component calling the use case to the use case component itself. + +##### Simpler boundaries + +An example of a simpler boundary is the [Facade pattern](https://en.wikipedia.org/wiki/Facade_pattern). Here, you take a complex subsystem and put it behind a class that exposes a nice interface to that subsystem. This does not provide the same amount of separation as the use of abstractions and does not allow you to control the direction of dependencies, but it can help with managing complexity and potentially prepare for the creation of a full boundary. + +![Clean Architecture boundary](_img/Architectural-boundaries/facade-pattern.jpg) + +#### Dynamically-linked deployable components + +- Parts separated by the boundaries are separately developable and deployable components + - Example: DLL or JAR files +- Deployed independently, but they still run in the same address space + - Communication still through simple method calls +- When different components are developed independently from each other, you need some kind of versioning and release management system that allows developers depending on a component to decide if and when to upgrade to its next version +- Dependencies between components need to be managed carefully in order to prevent dependency cycles + - See Dependency Inversion Principle (from the [SOLID principles](./oo-design/SOLID-principles.md)) + +See also [Deployable components](./Deployable-components.md) + +#### Local processes + +- Separate parts deployed as local processes +- Still live on the same machine, but they do not share the same address space (unless some memory sharing involved) + - Inter-process communication through shared memory, sockets or potentially some OS-specific mechanisms + - Context switching between processes (and potential marshalling and unmarshalling) means that the communication between processes has more overhead than just simple method calls. Where possible, unnecessary back-and-forth should be avoided. + +#### Services + +- Strongest, highest-level boundary +- Different services are assumed to live on different machines and communicate only over the network + - Communication often happens over HTTP or some kind of message queue + - Communication between services is expensive from a performance point of view +- Each service typically developed and operated by a separate team that takes ownership of he service, including its tech stack and data + - Sharing of a database between services is generally considered bad practice, because it prevents services from independently making changes to their database structure (or independently choosing the database technology which makes the most sense for the service) +- Freedom and flexibility: Ideally, changes to a service, except for its communication with other services, do not have any effect on other services +- Still some coupling! The fact that services communicate with each other means that services will still depend on each other to some extent + - Example: If your service needs customer data, there are scenarios where a change to the Customer service could impact you. + - These dependencies should be carefully managed + +See also [Microservices](reference-architectures/Microservices.md) + +### Combining different kinds of boundaries + +There is no need to choose only one kind of boundary! Different kinds of boundaries can be useful at different levels of your architecture. + +Example: You could have a set of microservices which you have obtained using vertical slicing. However, each of those microservices could have a layered architecture using horizontal slicing to separate different technical parts, either through source-level boundaries or as separately deployable components. + +## The costs of boundaries + +Benefits of boundaries do not come for free! + +- Potentially some performance impact +- Most costly impact: development and maintenance effort! + - Boundaries need to be developed and maintained + - Boundaries' decoupling mechanisms can increase the complexity of the system as a whole + +If you have five teams working on a system, they will likely benefit from having five clearly separated parts with stable interfaces connecting them. The same architecture could be harmful to productivity if there is only a single small team working on the system. The experience and knowledge of different team members also plays a part. + +When in doubt, keep it simple! + +- If there is no clear need for a boundary, it is likely that adding the boundary would be a case of over-engineering. +- Already plenty of horror stories about systems with so many layers of abstraction that it is almost impossible to figure out where certain logic sits in the codebase or where a certain new feature should be implemented + +Main conclusion: You have to make a tradeoff between the benefits and costs of each boundary instead of just blindly introducing boundaries and abstraction everywhere + +## Evolving boundaries + +Deciding on boundaries requires careful consideration: + +- Boundaries are expensive +- Introducing a new boundary which was not there before is typically very expensive + +However, it is impossible to know everything beforehand when building a system: + +- Context and requirements for the system are likely to change throughout its lifetime +- Likely impossible to foresee all technical challenges + +This means that the architecture of the system and the boundaries defining it will need to evolve along with the system itself. + +Some things that may need to evolve: + +- The location of the boundaries + - It’s possible that, as the system and the team grows, additional boundaries are needed to be able to maintain productivity + - On the other hand, the cost of maintaining certain boundaries may no longer outweigh the benefits they bring +- The separation mechanism used by a boundary + - An application could start as a monolith with some well-placed source-level boundaries, but over time it could make sense to start breaking up different parts into separate components or even separate services + - Ideally, a boundary should allow you to move to a higher (or lower) level of separation without the majority of the code having to know anything about the change + +A good architect will keep on watching the system for signs of parts that need additional separation or boundaries that have become less relevant. They will then make the necessary adjustments, taking into account both the benefits and costs associated with changing boundaries. This way, the architecture of the system will keep on evolving to suit the needs of the system and team. + +## Boundaries and the database + +### Boundaries between the domain and the database + +Typically makes sense to draw a boundary between the actual domain logic and the database (unless your application is a thin layer around the database that doesn’t really have any domain logic) + +One widespread convention: *Repository* pattern: + +- All interaction with the database is encapsulated inside Repository classes +- The domain logic interacts with these classes, without having to know anything database-specific + +```java +interface UserRepository { + getUsers(): Promise; + getUser(id: string): Promise; + saveUser(user: User): Promise; + deleteUser(id: string): Promise; +} + +class SqlServerUserRepository implements UserRepository { + // implement UserRepository methods by talking to SQL Server +} + +class UserService { + constructor(private repository: UserRepository) { } + + async updateName(id: string, newName: string) { + const user = await this.repository.getUser(id); + user.setName(newName); + await this.repository.saveUser(user); + } +} +``` + +If the domain logic is using the repository interface, then it also becomes easy to swap out the `SqlServerUserRepository` for a different implementation, for example an in-memory repository for testing purposes. + +```java +class InMemoryUserRepository implements UserRepository { + // implement UserRepository methods using in-memory storage +} +``` + +### Separation at the database level + +For larger systems, it can make sense to separate different parts of the application down to the database level. Each part uses different tables or a different database, with no links between data belonging to different parts. This kind of separation is considered good practice when setting up a microservices architecture. You can also do this in monolithic applications, potentially as a stepping stone towards a feature microservices architecture. + +Separation at the database level makes it easier to reason about separate parts of the application without having to think about other parts. It also provides more flexibility to change the schema or database technology for a certain part of the system. + +When drawing boundaries down to the database level, some data that is relevant to two parts of the system might exist on both sides of the boundary between them + +See also [Microservices](reference-architectures/Microservices.md) + +## Boundaries and the web + +### Decoupling the domain from the web + +One boundary that almost always makes sense to draw is the separation between your domain and the actual user interface that the user interacts with + +Typical to see `Controller` classes (some other terms are used as well) that take care of the interaction with the user and delegate all real work to the code implementing the actual business logic + +In principle, none of your business logic should be aware of how it is shown to the user, including whether or not the UI is a web UI! + +### Different representations of objects + +Different parts of the system have different goals -> they may also need different representations of the same object! + +- When doing server-side rendering, it often makes sense to have a separate view model that simply holds the data to be shown + - Data in the view model could be a transformed version of the data obtained from the domain model (e.g. formatting a date) or could aggregate data from several domain objects +- Data returned from the API could have a different format or structure than the actual domain objects inside the business logic part of the application + - Ideally, the data returned from APIs (or expected by APIs) will be aligned with what the consumers of the API care about +- If your frontend is a single-page application getting data from the backend over an API, feel free to create separate representations of that data that are more comfortable for the rest of the frontend to work with + - Backend is mostly about having consistent information regarding the objects in the system, while frontend is mostly about providing the user with an easy way of interacting with those objects + +### Micro frontends + +Even in a microservices architecture, it is typical to have a backend consisting of several microservices but a single monolithic frontend on top of it + +An alternative are micro frontends, where the frontend is stitched together from multiple independently maintained parts. See [Microservices](reference-architectures/Microservices.md). + +## Boundaries and libraries/frameworks + +Take care not to let too much of your code depend on third-party code! + +- External dependencies evolve in a way you do not control + - Their newest version including some critical bugfixes may introduce breaking changes in an API you use or even remove the functionality you use + - They may stop being properly maintained +- Your own requirements relevant to the dependency may change +- All of this can force you to change the way you use the dependency or even replace it with another dependency + +Consider creating a boundary around the external dependency that decouples the rest of the system from it (this boundary is sometimes called an *Anti-Corruption Layer*): + +- The public interface of that boundary should be written in terms of what your system needs from the dependency +- Logic inside the boundary will be specific to the interaction with that particular dependency. +- Benefits: + - If the API of the dependency changes or you replace it, the boundary protects you from having to change all code that used the dependency. As long as you can fulfill the contract specified by the public interface of the boundary, no code outside of the boundary has to be aware of the change. + - Especially useful if you consider the dependency to be a temporary solution that is sufficient for now but will most likely need to change in the future. The boundary allows you to avoid premature complexity by going for a simple solution, while keeping your options open regarding the upgrade to a more complex solution. + - Your can also use the boundary to create some automated test for the specific functionality that your system needs to get from the boundary. By testing against the boundary, you don’t have to change your tests in order to be able to test a new version of the dependency or even a replacement. + +Be extra careful when dealing with frameworks! + +- Frameworks tend to dictate the structure of your application and may even ask you to base your domain objects on the abstractions they provide. If you allow this, it will be very difficult to get the framework out afterwards. +- Could help to let the framework operate on some kind of separate representation of your domain objects instead of the domain objects themselves. Your boundary could then take care of performing the necessary translations between that separate representation and the actual domain objects. + +## Boundaries and duplication + +### False duplication + +Watch out for *false duplication*! + +- Real duplication: duplicates always have to change together + - This is what the DRY (Don’t Repeat Yourself) principle wants you to avoid +- False duplication: code/structures/... that are identical now but likely to change at different times or for different reasons + - Common with vertical slicing, where certain functionalities may start out looking similar but end up diverging significantly + - Can also happen with horizontal slicing, for example the apparent duplication between a database row and the corresponding structure we send to the UI + - It may be tempting to pass the database row directly to the UI, and in some cases this can be a good idea, but it isn’t hard to imagine that the structure of the data to show in the UI and the structure of the data in the DB could have to change independently of each other + - The fact that two things are the same at this moment does not necessarily mean that they are real duplicates and that that apparent duplication is a bad thing + - Attempts to get rid of false duplication tend to lead to unnecessary coupling through shared code, which will then come back to bite you when the “duplicates” suddenly need to change independently of each other + +### Data duplication and bounded contexts + +See [Microservices](reference-architectures/Microservices.md) + diff --git a/architecture-design/Deployable-components.md b/architecture-design/Deployable-components.md new file mode 100644 index 0000000..6c8b4d4 --- /dev/null +++ b/architecture-design/Deployable-components.md @@ -0,0 +1,142 @@ +# Deployable components + +See: + +- Clean Architecture (book by Robert C. Martin) +- [Clean Architecture – Components and Component Cohesion](https://www.codingblocks.net/podcast/clean-architecture-components-and-component-cohesion/) + +## Basic idea + +Deployable components: + +- Smallest parts that can be deployed separately +- Think JAR files, DLL files, ... +- Ways of deployment + - Could get linked into single executable + - Could be aggregated into single archive (example: .war file) + - Could be independently deployed as some kind of dynamically loaded plugins +- Regardless of how they end up being deployed, in principle they are independently deployable and thus independently developable + - Example: Executable is built from component A, B and C where both A and B depend on C. Because they are indepedently deployable components, new versions of C might be developed while still including an older version of C in the executables until A and B are adjusted to those new versions. + +## Component cohesion + +Answer to the question "which classes belong in which components?" + +Three important principles: + +- **REP:** Reuse-Release Equivalence Principle + - "The granule of reuse is the granule of release" + - Classes and modules within a release should be releasable together + - There must be some overarching theme or purpose that everything within a release shares + - The fact that classes and modules in a release share the same version numbering and same release documentation should make sense to both the author and the users + - Releases of shareable software components/modules must produce the proper notifications and release documentation + - Release numbers are important for measuring compatibility and for communicating changes + - See also [Semantic Versioning](https://semver.org/) + - breaking.new-features.fixes + - This principle is inclusive: tends to make components larger +- **CCP**: Common Closure Principle + - "Components should consist of classes that change for the same reason and at the same times" + - Similar to Single Responsibility Principle in [SOLID principles](oo-design/SOLID-principles.md) + - If the code of an application must change, you would ideally have all the changes to occur in a single component, so you only have to redeploy that single component + - Other components depending on that single component do not need to be revalidated or redeployed + - Relation to Open-Closed Principle from [SOLID principles](oo-design/SOLID-principles.md): group together the classes that are closed to the same type of changes + - This principle is inclusive: tends to make components larger +- **CRP**: Common Reuse Principle + - "Don’t force users of a component to depend on things that they don’t need" + - Similar to Interface Segregation Principle from [SOLID principles](oo-design/SOLID-principles.md) + - If your component has a dependency on another component, any time that depended upon component is changed, your component will need to be revalidated/recompiled + - This is true even if you don't care about the changes in that depended upon component + - Try to make sure that the classes in a component are inseparable, that it is almost impossible to depend on some but not on the others + - This principle is exclusive: tends to make components smaller + +These three tend to pull the answer to "which classes belong in which components?" in different directions + +Tension diagram (edges of diagram show the cost of abandoning the principle on the opposite vertex): + +![Tension diagram](_img/Deployable-components/cohesion-principles-tension-diagram.jpg) + +- Are “too many” components changing at the same time? Then your classes should be consolidated better (CCP) +- Do you have “too many” releases? You should take a look at streamlining and minimizing your dependencies (CRP) +- Is your code hard to re-use? You should improve your release artifacts (release numbers, documentation, notifications) (REP) + +> “A good architect finds a position in that tension triangle that meets the *current* concerns of the development team, but is also aware that those concerns will change over time.” – Robert C. Martin + +## Component coupling + +This is about the coupling that is caused by dependencies between components + +### Acyclic Dependencies Principle + +"When you draw a diagram of your components, using arrows to indicate dependencies, you should not have any cycles in that graph - the graph should be a Directed Acyclic Graph (DAG)" + +If the graph is indeed a DAG, it is easy to find out which components could be affected by a change in a certain component: just follow the dependency arrows backwards. Additionally, you can easily find a "topological ordering" for the graph which gives you the order in which to build the components so every component's dependencies are built before the component itself needs to be built. + +If you have a cycle, then the components in that cycle are not really independently developable and deployable! + +- People working on these components must make sure to all use exactly the same release of each other's components + - If A -> B -> C -> A, a new version of A must be compatible with a new version of C which is compatible with that new version of A +- Very difficult to test your component in isolation because your dependencies also depend on the changes you make +- Essentially, components in the cycle are reduced to a single component + +#### Breaking cycles + +Example cycle: A -> B -> C -> A + +Two primary mechanisms: + +- Applying the Dependency Inversion Principle from [SOLID principles](oo-design/SOLID-principles.md) to invert the direction of dependencies + - Example: Reverse dependency between A and C so we now have C <- A -> B -> C +- Creating a new component that two components can depend on instead of one depending on the other + - Example: Create new component D that A and C depend on so we now have D < - A -> B -> C -> D + +### Stability and volatility + +- Some components will be *volatile*: intended and expected to change + - Example: user interface +- Other components will be *stable*: intended and expected not to change (or not often) + - Example: core business logic + +Stability is closely related to the structure of dependencies! + +- If a component does not depend on any other components but several other components depend on it, it is likely to be stable + - No dependencies on others: not forced to change by others + - Others depending on the it: changes to the component require a lot of work because others likely need to be adjusted accordingly + - Special case common in statically typed languages: abstract components that contain nothing but some interfaces, don't depend on anything but are highly depended upon by others +- If a component depends on several other components but no other components depend on it, it is likely to be volatile + - Depends on several others: each of those others can force it to change + - No others depending on it: free to change without affecting others + +Possible metric for stability: # outgoing dependencies / (# incoming dependencies + # outgoing dependencies). Value is 0 if maximally stable, 1 if maximally volatile. + +#### Stable Dependencies Principle + +"You should depend on more stable components, not on more volatile ones" + +- If stable component depends on a volatile one, this means that the volatile one is expected to change often, which will force the stable component to become volatile as well +- If you design a component to be easy to change (volatile), all your efforts to make change easy become undone once someone starts depending on your component + +#### Stable Abstractions Principle + +"More stable components should also be more abstract components" + +Here, "abstractness" could be defined as the ratio of the number of classes in the component versus the number of those classes that are abstract classes or interfaces. Value = 0 if all concrete, 1 if all abstract. + +Reasoning: + +- The problem: It makes sense to have core business logic in a stable component that several others depend upon. However, the fact that it's hard to change could make overall architecture inflexible. How do we make a stable component flexible enough to change? +- The solution: use the Open-Closed Principle (from [SOLID principles](oo-design/SOLID-principles.md)) to create classes that are flexible enough to be extended without needing modification + - Originally aimed at inheritance, making classes easily extensible, which can often lead to making them abstract or even just defining an interface instead of a class + - Note: inheritance is not the only way to make a class extensible! We could also make the class support different kinds of behavior by passing in different objects (dependency injection, Strategy pattern). However, in that case, the objects to pass in will likely be specified by interfaces defined inside the component, again making the component more abstract. + +## Evolving the component structure + +Component structure doesn't need to reflect the different functional areas of the system! Instead, it reflects the buildability and maintainability of the application. + +Likely, top-down design of component structure will not work well! + +What typically works: + +- As the application grows, identify parts of the codebase where putting them in separate components would make development and maintenance easier +- Want to keep changes as localized as possible, so start paying attention to CCP +- As application continues to grow, more concern over creating reusable elements -> CRP +- As cycles appear, apply ADP, changing dependency directions or creating new components \ No newline at end of file diff --git a/architecture-design/_img/Architectural-boundaries/clean-architecture-boundary.jpg b/architecture-design/_img/Architectural-boundaries/clean-architecture-boundary.jpg new file mode 100644 index 0000000..87aa71c Binary files /dev/null and b/architecture-design/_img/Architectural-boundaries/clean-architecture-boundary.jpg differ diff --git a/architecture-design/_img/Architectural-boundaries/facade-pattern.jpg b/architecture-design/_img/Architectural-boundaries/facade-pattern.jpg new file mode 100644 index 0000000..070c5e7 Binary files /dev/null and b/architecture-design/_img/Architectural-boundaries/facade-pattern.jpg differ diff --git a/architecture-design/_img/Architectural-boundaries/strategy-pattern.png b/architecture-design/_img/Architectural-boundaries/strategy-pattern.png new file mode 100644 index 0000000..d124f90 Binary files /dev/null and b/architecture-design/_img/Architectural-boundaries/strategy-pattern.png differ diff --git a/architecture-design/_img/Deployable-components/cohesion-principles-tension-diagram.jpg b/architecture-design/_img/Deployable-components/cohesion-principles-tension-diagram.jpg new file mode 100644 index 0000000..b53e6a1 Binary files /dev/null and b/architecture-design/_img/Deployable-components/cohesion-principles-tension-diagram.jpg differ diff --git a/architecture-design/SOLID-principles.md b/architecture-design/oo-design/SOLID-principles.md similarity index 100% rename from architecture-design/SOLID-principles.md rename to architecture-design/oo-design/SOLID-principles.md diff --git a/architecture-design/_img/SOLID-principles/DIP-after.png b/architecture-design/oo-design/_img/SOLID-principles/DIP-after.png similarity index 100% rename from architecture-design/_img/SOLID-principles/DIP-after.png rename to architecture-design/oo-design/_img/SOLID-principles/DIP-after.png diff --git a/architecture-design/_img/SOLID-principles/DIP-before.png b/architecture-design/oo-design/_img/SOLID-principles/DIP-before.png similarity index 100% rename from architecture-design/_img/SOLID-principles/DIP-before.png rename to architecture-design/oo-design/_img/SOLID-principles/DIP-before.png diff --git a/architecture-design/_img/SOLID-principles/ISP-after.png b/architecture-design/oo-design/_img/SOLID-principles/ISP-after.png similarity index 100% rename from architecture-design/_img/SOLID-principles/ISP-after.png rename to architecture-design/oo-design/_img/SOLID-principles/ISP-after.png diff --git a/architecture-design/_img/SOLID-principles/ISP-before.png b/architecture-design/oo-design/_img/SOLID-principles/ISP-before.png similarity index 100% rename from architecture-design/_img/SOLID-principles/ISP-before.png rename to architecture-design/oo-design/_img/SOLID-principles/ISP-before.png diff --git a/architecture-design/_img/SOLID-principles/OCP-after-adding-functionality.png b/architecture-design/oo-design/_img/SOLID-principles/OCP-after-adding-functionality.png similarity index 100% rename from architecture-design/_img/SOLID-principles/OCP-after-adding-functionality.png rename to architecture-design/oo-design/_img/SOLID-principles/OCP-after-adding-functionality.png diff --git a/architecture-design/_img/SOLID-principles/OCP-before-adding-functionality.png b/architecture-design/oo-design/_img/SOLID-principles/OCP-before-adding-functionality.png similarity index 100% rename from architecture-design/_img/SOLID-principles/OCP-before-adding-functionality.png rename to architecture-design/oo-design/_img/SOLID-principles/OCP-before-adding-functionality.png diff --git a/architecture-design/_img/SOLID-principles/Observer.png b/architecture-design/oo-design/_img/SOLID-principles/Observer.png similarity index 100% rename from architecture-design/_img/SOLID-principles/Observer.png rename to architecture-design/oo-design/_img/SOLID-principles/Observer.png diff --git a/architecture-design/_img/SOLID-principles/SRP-after.png b/architecture-design/oo-design/_img/SOLID-principles/SRP-after.png similarity index 100% rename from architecture-design/_img/SOLID-principles/SRP-after.png rename to architecture-design/oo-design/_img/SOLID-principles/SRP-after.png diff --git a/architecture-design/_img/SOLID-principles/SRP-before.png b/architecture-design/oo-design/_img/SOLID-principles/SRP-before.png similarity index 100% rename from architecture-design/_img/SOLID-principles/SRP-before.png rename to architecture-design/oo-design/_img/SOLID-principles/SRP-before.png diff --git a/architecture-design/reference-architectures/Microservices.md b/architecture-design/reference-architectures/Microservices.md new file mode 100644 index 0000000..01daf11 --- /dev/null +++ b/architecture-design/reference-architectures/Microservices.md @@ -0,0 +1,113 @@ +# Microservices + +See: + +- Building Evolutionary Architectures (book by Neal Ford, Rebecca Parsons and Patrick Kua) ([summary slides](https://www.slideshare.net/thekua/building-evolutionary-architectures)) +- Clean Architecture (book by Robert C. Martin) +- [BoundedContext](https://www.martinfowler.com/bliki/BoundedContext.html) +- [Pattern: Database per service](https://microservices.io/patterns/data/database-per-service.html) +- [How to keep relationship integrity with Microservice Architecture](https://softwareengineering.stackexchange.com/questions/381279/how-to-keep-relationship-integrity-with-microservice-architecture) + +## Basic idea + +![Microservices](_img/Microservices/microservices.png) + +- The system is divided into several small services that each encapsulate a certain functional area across several layers of the technical stack, even down to the database + - "Shared nothing", decrease coupling between services as much as possible + - Sharing of a database between services is generally considered bad practice, because it prevents services from independently making changes to their database structure (or independently choosing the database technology which makes the most sense for the service) + - Freedom to choose or change the technology used by a service based on what makes most sense. Some service may use a relational database while another one uses a document store. A service providing information about the relationships between different users could switch to a graph database without any other service being affected by the change. + +- Different small teams each take ownership of one or more of these services + - Team responsible for a microservice takes control of development and deployment + - Each service is expected to handle reasonable error scenarios and recover if possible + - Changes within a single functional domain (= single service) can happen within a single team + - Coordination with other teams is only required if the communication with their services needs changes as well +- Services integrate with each other by passing messages, most often over HTTP or message queues +- Typically combined with Continuous Delivery, automatic machine provisioning and deployment, ... +- Monitoring and logging typically first-class architectural concepts + - Several services, each of which might be scaled across several instances -> large number of processes to watch + - This means having a good monitoring and logging setup is essential in keeping the system running smoothly + +## Coupling in a microservices architecture + +The goal of a microservices architecture is typically to minimize inappropriate coupling. However, some coupling is still needed to create a useful system. + +### Integration coupling + +This type of coupling is the most obvious one: The fact that services communicate with each other means that services will still depend on each other to some extent. If your service needs customer data, there are scenarios where a change to the Customer service could impact you. As always, these dependencies should be carefully managed. + +### Service template coupling + +*Service template*: shared template that all services can build upon + +- Contains general stuff like monitoring, logging, and authentication/authorization. + +- Teams maintaining the services build their services upon this template +- Much easier to ensure compliance and manage upgrades to monitoring system etc. than if each service team would build their own version of this + - When the infrastructure team pushes an upgrade to the service template, the services pick it up the next time they go through the deployment pipeline + +Example general service templates: [DropWizard]([https://www.dropwizard.io](https://www.dropwizard.io/)) and [Spring Boot](https://spring.io/projects/spring-boot) + +## Data duplication and bounded contexts + +What if different services are interested in the same concept? Remember, sharing databases between services is considered bad practice! + +Typical approach: each service stores its own representation of the concept + +This can be seen as duplication, but: + +- If a service stores the data it needs, it is less dependent on other services (both regarding availability and code coupling) +- Data that different services store for that same thing will depend on what the specific services need +- Same domain concept may have completely different representations in different services +- Data (or shape of data) regarding the same concept in different services will likely change at different times and for different reasons + +This approach corresponds to the DDD concept of [Bounded Contexts](https://www.martinfowler.com/bliki/BoundedContext.html) + +Example: e-commerce system using a microservices architecture + +- Situation: + - Customer service which manages a whole lot of data about each customer, including their shipping address + - Order service, which maintains orders and needs the shipping address of the customer +- Approach 1: let Order service store a customer ID with each Order + - Whenever all information for an order needs to be retrieved, customer’s address is retrieved from the Customer service + - Problem: if the Customer service is unavailable or the customer has been deleted, the Order service has no idea which address to use for the order + - Problem: if the customer’s address changes, the current shipping address for a customer may not be the address that a certain order was shipped to +- Approach 2: let Order service store customer ID and customer’s shipping address + - Essentially, Order service stores a snapshot of the data it needs + - If the order is retrieved from the Order service later on, the service will use the shipping address that it has stored + +Some services may even be entirely focused on aggregating bits of data from other services! + +- Can be useful for search, analysis, ... + +- Example: functionality to view some general info about the most recent orders of customers, allowing to filter customers by some key attributes + - Can make sense to put this in separate Insights service + - Service gathers data from the Customer and Order services (could be push, through API calls or message queue, or pull) + - Service stores data in a format that includes exactly the required information + - When the service receives a request, it can gather the required data directly from its own database + +### Micro frontends + +See: + +- [Micro Frontends](https://micro-frontends.org/) +- [Micro frontends—a microservice approach to front-end web development](https://medium.com/@tomsoderlund/micro-frontends-a-microservice-approach-to-front-end-web-development-f325ebdadc16) +- [MicroFrontends](https://martinfowler.com/articles/micro-frontends.html) + +Even with microservices in the backend, the frontend is often monolithic: + +![Monolithic frontend](_img/Microservices/monolithic-frontend.png) + +- Often a single, large and feature-rich single-page app cutting across all of the functional areas represented by the backend microservices +- Problems: + - Frontend can become so big that it’s difficult to maintain + - Frontend often also developed by a separate team, meaning that that team needs to coordinate with the backend teams when building functionality. + +Alternative approach: Micro frontends + +![Monolithic frontend](_img/Microservices/micro-frontend.png) + +- Frontend is split into different parts in the same way that the backend is +- Teams are responsible for their functional part of the application across the entire stack, from the database up to the frontend. +- The actual frontend that the user interacts with is stitched together from the functional parts developed by different teams + - Some patterns/frameworks available that can help \ No newline at end of file diff --git a/architecture-design/reference-architectures/_img/Microservices/micro-frontend.png b/architecture-design/reference-architectures/_img/Microservices/micro-frontend.png new file mode 100644 index 0000000..e6a9135 Binary files /dev/null and b/architecture-design/reference-architectures/_img/Microservices/micro-frontend.png differ diff --git a/architecture-design/reference-architectures/_img/Microservices/microservices.png b/architecture-design/reference-architectures/_img/Microservices/microservices.png new file mode 100644 index 0000000..9f63b5e Binary files /dev/null and b/architecture-design/reference-architectures/_img/Microservices/microservices.png differ diff --git a/architecture-design/reference-architectures/_img/Microservices/monolithic-frontend.png b/architecture-design/reference-architectures/_img/Microservices/monolithic-frontend.png new file mode 100644 index 0000000..2d7b3a2 Binary files /dev/null and b/architecture-design/reference-architectures/_img/Microservices/monolithic-frontend.png differ diff --git a/data/CAP-theorem.md b/data/CAP-theorem.md index 53ef1cb..eed8d40 100644 --- a/data/CAP-theorem.md +++ b/data/CAP-theorem.md @@ -4,8 +4,6 @@ See: - [CAP theorem](https://en.wikipedia.org/wiki/CAP_theorem) - Designing Data-Intensive Applications (book by Martin Kleppmann) -- [SQL Server Availability Modes](https://docs.microsoft.com/en-us/sql/database-engine/availability-groups/windows/availability-modes-always-on-availability-groups?view=sql-server-2017) -- [Offload read-only workload to secondary replica of an Always On availability group](https://docs.microsoft.com/en-us/sql/database-engine/availability-groups/windows/active-secondaries-readable-secondary-replicas-always-on-availability-groups?view=sql-server-2017) This theorem states that it is impossible for a distributed data store to simultaneously provide more than two out of the following three guarantees: @@ -39,7 +37,15 @@ See also [ACID properties](./sql/ACID.md) Both mean something completely different: - CAP consistency: Every read returns either the relevant value as it was written by the latest successful write or an error - - ACID consistency: The execution of the transaction must bring the database to a valid state, respecting the database’s schema -In fact, when relational databases are deployed in a distributed fashion, there are typically different modes available that can have an impact on CAP consistency. For example, when settings up a high-availability cluster for Microsoft SQL Server, you have the choice between the availability modes *synchronous commit* and *asynchronous commit*. Synchronous commit waits to return for a transaction until it has effectively been synchronized to the other instances (secondary replicas). Asynchronous commit, on the other hand, does not wait for the secondary replicas to catch up. If asynchronous commit is used and the cluster is configured to allow reads to go directly to the secondary replicas, it is possible that reads return stale data. \ No newline at end of file +In fact, when relational databases are deployed in a distributed fashion, there are typically different modes available that can have an impact on CAP consistency. + +### SQL Server example + +See: + +- [SQL Server Availability Modes](https://docs.microsoft.com/en-us/sql/database-engine/availability-groups/windows/availability-modes-always-on-availability-groups?view=sql-server-2017) +- [Offload read-only workload to secondary replica of an Always On availability group](https://docs.microsoft.com/en-us/sql/database-engine/availability-groups/windows/active-secondaries-readable-secondary-replicas-always-on-availability-groups?view=sql-server-2017) + +When setting up a high-availability cluster for Microsoft SQL Server, you have the choice between the availability modes *synchronous commit* and *asynchronous commit*. Synchronous commit waits to return for a transaction until it has effectively been synchronized to the other instances (secondary replicas). Asynchronous commit, on the other hand, does not wait for the secondary replicas to catch up. If asynchronous commit is used and the cluster is configured to allow reads to go directly to the secondary replicas, it is possible that reads return stale data. \ No newline at end of file diff --git a/data/sql/Normalization.md b/data/sql/Normalization.md index 430d4a9..e720507 100644 --- a/data/sql/Normalization.md +++ b/data/sql/Normalization.md @@ -62,11 +62,11 @@ Solution: make Subject into its own table **Book - Subject table** (many-to-many relationship, needed because book can have multiple subjects and multiple books can share the same subject) -| Title | ***Subject ID*** | -| :----------------------------------------------- | :--------------- | -| Beginning MySQL Database Design and Optimization | *1* | -| Beginning MySQL Database Design and Optimization | *2* | -| Beginning MySQL Database Design and Optimization | 3 | +| Title | Subject ID | +| :----------------------------------------------- | :--------- | +| Beginning MySQL Database Design and Optimization | 1 | +| Beginning MySQL Database Design and Optimization | 2 | +| Beginning MySQL Database Design and Optimization | 3 | ### 2NF (second normal form) diff --git a/java/Equals.md b/java/Equals.md index ea20bc1..79324dc 100644 --- a/java/Equals.md +++ b/java/Equals.md @@ -259,7 +259,7 @@ What if we want to include the color in the `equals` method so that a `ColorPoin If we want this, we have to accept that a `ColorPoint` will never be equal to any `Point`. The reason for this is transitivity (see above). If we want to say that `ColorPoint(1, 1, Color.RED)` and `ColorPoint(1, 1, Color.BLUE)` are both equal to `Point(1, 1)` , then transitivity would imply that they are also equal to each other. That is exactly what we didn't want here. -This could be seen as a violation of the [Liskov substitution principle](../architecture-design/SOLID-principles.md) +This could be seen as a violation of the [Liskov substitution principle](../architecture-design/oo-design/SOLID-principles.md) ```java @Test