Skip to content

Latest commit

 

History

History
995 lines (688 loc) · 64.8 KB

README.md

File metadata and controls

995 lines (688 loc) · 64.8 KB
Raw

Table of Contents

Software Architecture Mindmap

Software terminologies and concepts, software architecture overview

Summarized the keywords and solutions have faced in my learning and experience.

  • Full Version

Software_Architecture_Mindmap.png

Three main pillars upon software architecture

  • Modern Application Development

  • Cloud Computing (AWS/Azure/GCP)

  • Data Science (ML/NN)

and

Numerous technologies and methodologies.

ⓒ 2022. (https://github.com/kimtth) all rights reserved.

This mindmap created by https://app.mindmapmaker.org/

Software Architecture Reference

  • Google SRE Handbook

    Expand

    🔹 Latency is the response time of your application, usually expressed in milliseconds

    🔹 Throughput is how many transactions per second or minute your application can handle

    🔹 Errors is usually measured in a percent of

    🔹 Saturation is the ability of your application to use the available CPU and Memory

  • InfoQ minibooks: Architectures You’ve Always Wondered About .. 2021 / 2023 / 2024 / Cell-Based Architecture

Building from scratch

Cloud Architecture

Expand 
🔹 Abstractly speaking, a landing zone helps you plan for and design an Azure deployment, by conceptualizing a designated area for placement and integration of resources. 

There are two types of landing zones:

1. `platform landing zone`: provides centralized enterprise-scale foundational services for workloads and applications.
2. `application landing zone`: provides services specific to an application or workload.

Computer Science courses

Industry Trends

Newsletter

Tools & Research

Engineering blog

Other Topics

Computer Science Books

  1. General

  2. Good Practices

  3. Data Structures and Algorithms

  4. Data

  5. Testing

  6. Software Architecture

  7. Distributed Systems

  8. DevOps

  9. Machine Learning

Computer Science Papers

Computer Science Papers Every Developers Should Read

  1. On the Criteria To Be Used in Decomposing Systems into Modules (1972): D.L. Parnas
  2. An Axiomatic Basis for Computer Programming (1969): C.A.R. Hoare
  3. Time, Clocks, and the Ordering of Events in a Distributed System (1978): L. Lamport
  4. Out of the Tar Pit (2006): B. Moseley, P. Marks
  5. Dynamo: Amazon’s Highly Available Key-value Store (2007): G. DeCandia et al.
  6. MapReduce: Simplified Data Processing on Large Clusters (2004): J. Dean, S. Ghemawat
  7. A Note On Distributed Computing (1994): J. Waldo, G. Wyant, A. Wollrath, S. Kendall
  8. A Metrics Suite for Object-Oriented Design (1994): S.R. Chidamber
  9. A Relational Model of Data for Large Shared Data Banks (1969): E.F. Codd
  10. Why Functional Programming Matters (1990): J. Hughes

Distributed Systems!

25 Papers That Completely Transformed the Computer World

  • ref [May 2024]
  1. Dynamo: Amazon’s Highly Available Key Value Store
  2. Google File System: Insights into a highly scalable file system
  3. Scaling Memcached at Facebook: A look at the complexities of caching
  4. BigTable: The design principles behind a distributed storage system
  5. Borg: Large Scale Cluster Management at Google
  6. Cassandra: A look at the design and architecture of a distributed NoSQL database
  7. Attention Is All You Need: Into a new deep learning architecture known as the transformer
  8. Kafka: Internals of the distributed messaging platform
  9. FoundationDB: A look at how a distributed database works
  10. Amazon Aurora: How Amazon provides high availability and performance
  11. Spanner: Design and architecture of Google’s globally distributed database
  12. MapReduce: A detailed look at how MapReduce enables parallel processing of massive volumes of data
  13. Shard Manager: Understanding the generic shard management framework
  14. Dapper: Insights into Google’s distributed systems tracing infrastructure
  15. Flink: A detailed look at the unified architecture of stream and batch processing
  16. A Comprehensive Survey on Vector Databases
  17. Zanzibar: A look at the design, implementation, and deployment of a global system for managing access control lists at Google
  18. Monarch: Architecture of Google’s in-memory time series database
  19. Thrift: Explore the design choices behind Facebook’s code-generation tool
  20. Bitcoin: The ground-breaking introduction to the peer-to-peer electronic cash system
  21. WTF - Who to Follow Service at Twitter: Twitter’s (now X) user recommendation system
  22. MyRocks: LSM-Tree Database Storage Engine
  23. GoTo Considered Harmful
  24. Raft Consensus Algorithm: Learn about the more understandable consensus algorithm
  25. Time Clocks and Ordering of Events: The extremely important paper that explains the concept of time and event ordering in a distributed system

Data Science (ML/NN)

30 free e-books

  1. Deep Learning - Ian Goodfellow, Yoshua Bengio, and Aaron Courville
  2. Mathematics for Machine Learning - Marc Peter Deisenroth, A. Aldo Faisal, Cheng Soon Ong
  3. An Introduction to Statistical Learning - Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani, Jonathan Taylor
  4. The Elements of Statistical Learning - Jerome H. Friedman, Robert Tibshirani, and Trevor Hastie
  5. Probabilistic Machine Learning: An Introduction - Kevin Patrick Murphy
  6. Probabilistic Machine Learning: Advanced Topics - Kevin Patrick Murphy
  7. Understanding Machine Learning - Shai Shalev-Shwartz and Shai Ben-David
  8. Automated Machine Learning - Frank Hutter, Lars Kotthoff, Joaquin Vanschoren
  9. Applied Causal Inference - Uday Kamath, Kenneth Graham, Mitchell Naylor
  10. Reinforcement Learning: An Introduction - Richard S. Sutton and Andrew G. Barto
  11. The Hundred-Page Machine Learning Book - Andriy Burkov
  12. Machine Learning Engineering - Andriy Burkov
  13. Natural Language Processing with Python - Steven Bird, Ewan Klein, and Edward Loper
  14. Dive into Deep Learning - Aston Zhang, Zachary C. Lipton, Mu Li, Alexander J. Smola
  15. Machine Learning Yearning - Andrew NG
  16. Machine Learning for Humans - Vishal Maini, Samer Sabri
  17. Pattern Recognition and Machine Learning - Christopher M. Bishop
  18. Deep Learning on Graphs - Yao Ma and Jiliang Tang
  19. Approaching (Almost) Any Machine Learning Problem - Abhishek Thakur
  20. Feature Engineering and Selection - Max Kuhn and Kjell Johnson
  21. Hands-On Machine Learning with R - Bradley Boehmke & Brandon Greenwell
  22. Deep Learning Interviews - Shlomo Kashani and Amir Ivry
  23. Machine Learning Interpretability - Patrick Hall and Navdeep Gill
  24. Interpretable Machine Learning - Christoph Molnar
  25. Boosting: Foundations and Algorithms - Robert E. Schapire, Yoav Freund
  26. A Brief Introduction to Machine Learning for Engineers - Osvaldo Simeone
  27. Speech and Language Processing - Daniel Jurafsky & James Martin
  28. Computer Vision: Models, Learning, and Inference - Simon J.D. Prince
  29. Information Theory, Inference and Learning Algorithms - David J. C. MacKay
  30. Machine Learning For Dummies - Judith Hurwitz and Daniel Kirsch
  31. Algebra, Topology, Differential Calculus, and Optimization Theory for Computer Science and Machine Learning

github

  1. The Data Engineering Handbook
  2. Machine Learning for Beginners
  3. Machine Learning YouTube Videos
  4. Mathematics for Machine Learning
  5. Deep Learning Book
  6. Machine Learning ZoomCamp
  7. Machine Learning Tutorials
  8. Awesome Machine Learning
  9. CS 229 Machine Learning Cheatsheets
  10. Machine Learning Interview Guide
  11. Awesome Production Machine Learning
  12. 365 Data Science Flashcards
  13. ref > Virgilio | Python Data Science Handbook | Microsoft: 10 Weeks, 20 Lessons, Data Science | Data science Python notebooks | 📚 Papers & tech blog | Open Source Data Science Masters | Awesome Data Science | Data science interview questions and answers | free self-taught education in Data Science!
  14. data science project template
  15. labml.ai Deep Learning Paper Implementations: 60+ Implementations/tutorials of deep learning papers with side-by-side notes
  16. Deep Learning Models: A collection of various deep learning architectures, models, and tips
  17. fastai book: The fastai book, published as Jupyter Notebooks

Terminology and Comparisons

  1. Gartner's PACE Layered Application Strategy: A methodology for categorizing, selecting, managing and governing applications based on their characteristics and the speed of change they require1.

  2. JIT vs AOT: JIT and AOT are two types of compilers that differ in when they convert a program from one language to another, either at run-time or build-time.

  3. SSG: Static site generator list: A tool that generates a full static HTML website based on raw data and a set of templates.

  4. Popular Enterprise Architecture Frameworks: TOGAF, Zachman, Federal Enterprise Architecture (FEA), Gartner Enterprise Architecture Framework, Business Architecture Guild’s BIZBOK, Department of Defense Architecture Framework (DoDAF), ArchiMate, and Sherwood Applied Business Security Architecture (SABSA).

  5. Are Architecture Styles, Patterns, and Design Patterns Different?

    Architecture Styles vs Patterns vs Design Patterns 
     𝟭. 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗮𝗹 𝘀𝘁𝘆𝗹𝗲𝘀

 This is the highest level of abstraction, where architectural designs instruct us on structuring our code. The highest level of granularity describes the application's layers and high-level modules and how they relate to and interact with one another. Examples of architectural styles include:
 🔹 Monolith
 🔹 Layered
 🔹 Event-driven
 🔹 Self-contained Systems
 🔹 Microservices
 🔹 Space-Based

 𝟮. 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗮𝗹 𝗽𝗮𝘁𝘁𝗲𝗿𝗻𝘀

 These patterns represent a way to implement an architectural style, so we can do this regularly. Some examples are how to separate the user interface (UI) and data, how internal modules interact, and what layers we will use. Patterns answer these types of questions. They usually impact the code base and how to structure the code inside. Examples of architectural patterns include:

 🔹 Model-View-Presenter (MVP): 1:1 Relationship between View and Presenter. e.g., Windows forms
 🔹 Model-View-Controller (MVC): e.g., Smalltalk, ASP.Net MVC
 🔹 Model–View–Viewmodel (MVVM): One to Many relationship between View and ViewModel. e.g., Silverlight, WPF, AngularJs: 
 🔹 Domain-Driven Design

 𝟯. 𝗗𝗲𝘀𝗶𝗴𝗻 𝗽𝗮𝘁𝘁𝗲𝗿𝗻𝘀

 These differ from architectural patterns in that they focus on a smaller code base area and have a smaller influence (focus on a local problem). These include limiting the creation of a class to only one object or notifying all dependent objects when the internal state of an object is changed. These patterns are described in the book "Design Patterns: Elements of Reusable Object-Oriented Software" by Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides from 1994.

 We have 𝘁𝗵𝗿𝗲𝗲 𝗴𝗿𝗼𝘂𝗽𝘀 𝗼𝗳 𝗗𝗲𝘀𝗶𝗴𝗻 𝗣𝗮𝘁𝘁𝗲𝗿𝗻𝘀:

 🔹 𝗖𝗿𝗲𝗮𝘁𝗶𝗼𝗻𝗮𝗹: here we have Factory Method, Builder, Singleton, ...
 🔹 𝗦𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗮𝗹: here we have an Adapter, Bridge, and Decorator, ...
 🔹 𝗕𝗲𝗵𝗮𝘃𝗶𝗼𝗿𝗮𝗹: here we have Command, Iterator, State, Strategy, ...

  6. Memory consistency model: A Primer on Memory Consistency and Cache Coherence

    SC vs TSO vs Relaxed Memory Consistency 
     𝟭. Sequential Consistency (SC): Operations execute in order as per the program.
 🔹 SC preserves order for two memory operations from the same thread for all four combinations of loads and stores (Load → Load, Load → Store, Store → Store, and Store → Load). 
 🔹 MIPS R10000

 𝟮. Total Store Order (TSO): Reads can happen before preceding writes complete. 
 🔹 TSO preserves the first three orders (Load → Load, Load → Store, Store → Store) but not Store → Load order.
 🔹 x86 CPU.

 𝟯. Relaxed Memory Consistency: Allows more reordering of operations for performance. 
 🔹 ARM and RISC-V

  7. API Gateway vs Load Balancer

    Expand

    🔹 API Gateway: Manages access to backend services, handles tasks like rate-limiting, authentication, logging, and security policies.

    🔹 Load Balancer: Distributes network traffic across multiple servers for high availability and even load distribution.

  8. Data engineering & Data Scientists Vocab 101 ref / 15 DS/ML Cheat Sheets

    Expand 🔹 Data engineering Vocab 101

    ref

    Data engineering 101

    🔹 75 Key Terms That Data Scientists Remember by Heart

    ref

    Data engineering 01

    🔹 A Comprehensive NumPy Cheat Sheet Of 40 Most Used Methods

    ref

    Data engineering 02

    🔹 15 Pandas ↔ Polars ↔ SQL ↔ PySpark Translations

    ref

    Data engineering 03

    🔹 11 Key Probability Distributions

    ref

    Data engineering 04

    🔹 6 Must-Know Types of Clustering Algorithms in Machine Learning

    ref

    Data engineering 05

    🔹 25 Most Important Mathematical Definitions in Data Science

    ref

    Data engineering 06

    🔹 10 Regression and Classification Loss Functions

    ref

    Data engineering 07

  9. Transfer Learning, Fine-tuning, Multitask Learning and Federated Learning ref

    Expand -

  10. DevOps, Platform engineering and SRE (site reliability engineering) ref

    SRE vs. DevOps vs. Platform Engineering

    🔹DevOps, SRE, and Platform Engineering are practices that streamline software development and maintenance. They all involve automation and collaboration.

    🔹DevOps covers the entire software development process promoting team collaboration.

    🔹SRE focuses on system reliability, including application monitoring and emergency response.

    🔹Platform Engineering manages the infrastructure and tools needed for software development and operations.

    🔹DevOps is about the whole development process, SRE emphasizes reliability and scalability, and Platform Engineering is about infrastructure and tool management.

  11. API Protocols (ref. ByteByteGo)

    Expand api

  12. Web services and APIs (SOAP, RestAPI, GraphQL, gRPC and Kafka) ref

    SOAP, RestAPI, GraphQL, gRPC and Kafka

    🔹SOAP (Simple Object Access Protocol): XML-based protocol for web services, heayweight, favored for security and reliability.

    🔹REST (Representational State Transfer): Uses HTTP methods, simple and easy to use, but can be resource-heavy.

    🔹GraphQL: Allows flexible data queries, reduces data over-fetching.

    🔹gRPC (Google Remote Procedure Call)**: High-performance RPC framework, ideal for connecting microservices. Built on top of HTTP/2 and uses Protocol Buffers for data exchange.

    🔹Kafka: Distributed streaming platform, uses publish-subscribe model for message queueing. real-time consistency. "at-least-once" delivery.

  13. Real-time communication and messaging (MQTT, AMQP and WebSocket) ref

    MQTT vs AMQP vs WebSocket

    🔹MQTT (Message Queuing Telemetry Transport): Lightweight messaging protocol, uses publish-subscribe model, ideal for IoT and M2M communication. Three levels of Quality of Service (QoS): “At most once” (QoS 0), “At least once” (QoS 1), and “Exactly once” (QoS 2).

    🔹AMQP (Advanced Message Queuing Protocol): Open-standard application layer protocol, robust message delivery, routing, and security features. Two qualities of service: “At most once (delivered once or lost)” and “At least once (delivered one or more times.)”.

    🔹WebSocket: Enables full-duplex communication channels over a single TCP connection

  14. Reactive programming vs event-driven architecture ref

    • Event-Driven: Handles user actions or system events. More general and can be used in any context where an event occurs
    • Reactive: Data-driven approach. managing data streams and propagating changes, like in a spreadsheet model.

  15. RABC vs ReABC: RBAC (Role-Based Access Control) is an authorization model that assigns permissions based on predefined roles. On the other hand, ReBAC (Relationship-Based Access Control) extends RBAC’s capabilities by considering relationships between entities.

  16. Conway's law: Software engineering principle that states that the structure of a system reflects the structure of the organization that designs it.

  17. Data Management in Distributed systems (Partitioning, Shuffling and Bucketing)

    Partitioning vs Shuffling vs Bucketing

    🔹Partitioning: The process of dividing a large dataset into smaller parts, known as partitions. This process splits Hive table's files into multiple files. For example, ../hive/warehouse/sales_table/product_id=P1.

    🔹Shuffling: Shuffling is the process of redistributing data across different partitions. The overhead of operations can be ranked as follows: orderby > join > groupby.

    🔹Bucketing: This is the process of decomposing data into manageable parts based on a certain column, thereby improving query performance and storage efficiency. It is best used when there are very few repeating values in a column (for example 1. a primary key column). For instance, Bucket0: ../hive/warehouse/sales_table/product_id=P1/000000_0, Bucket1: ../hive/warehouse/sales_table/product_id=P1/000001_0, and so on.

  18. SSO (Single Sign-On) is an authentication scheme that allows a user to log in with a single ID and password to any of several related, yet independent, software systems.

    SSO workflow, Types of SSO, SSO Implementations

    🔹SSO workflow: Identity Provider (IdP), Service Provider (SP), SSO Server

    • IdP: Central Authentication server e.g., Google
    • SP: Individual Applications rely on SSO e.g, Trello
    • SSO Server: Bridge between IdP and SPs

    🔹Types of SSO: SAML, OAuth (Open Authorization) 2.0, Open ID Connect (OIDC)

    Protocol Purpose Token Format -
    OAuth 2.0 Open standard for Authorization Access Tokens Temporary access to 3rd party app
    OpenID Connect (OIDC) Open standard for Authentication JSON Web Token (JWT) Newer type of SSO based on OAuth 2.0, Straightforward protocol than SAML
    SAML Authentication, Authorization XML Most common, Use SAML Protocol to exchange authentication between SSO server and SP

    🔹Some other Types of SSO: Kerberos, Smart card authentication

    • Kerberos: Less suitable for internet-facing SSO due to the shared secret between KDC (Key Distributin Center) and all participants.
    • Smart card authentication: Physical card

    🔹SSO Implementations: Microsoft Entra ID (FKA Micorsoft Active Directory), Okta, Ping Identity, OneLogin, Auth0

  19. Deployment Styles: Blue/Green, Canary, and A/B

    Blue/Green, Canary, A/B

    🔹Blue/Green Deployment: Two identical environments, "Blue" and "Green". Deploy new version in inactive environment, test, then switch users to it. For example, AWS supports blue/green deployment strategies including Elastic Beanstalk, OpsWorks, CloudFormation, CodeDeploy, and Amazon ECS.

    🔹Canary Deployment: Roll out new version to a small group of users, monitor feedback, then do a full-scale release.

    🔹A/B Testing: Compare two versions of a webpage or app to see which performs better. A typical example of A/B testing is website usability testing.

  20. Flaky Test: A Flaky Test is a test that sometimes passes and sometimes fails, despite no changes in the code. Causes can include poorly written tests, async waits, test order dependency, and concurrency issues. They can slow down CI/CD pipelines and cause issues for end users. ref

  21. Hadoop Ecosystem

    Hadoop vs Azure, AWS, GCP

    🔹1. HDFS (File Storage): Azure Data Lake Storage, Amazon S3, Google Cloud Storage

    🔹2. YARN (Resource Management): No direct equivalent in Azure, AWS, GCP

    🔹3. MapReduce (Data Processing): HDInsight, Amazon EMR, Google Cloud Dataproc

    🔹4. Spark (Fast Data Processing): Databricks, Spark in HDInsight, Azure Synapse Analytics, Amazon EMR, Google Cloud Dataproc

    🔹5. PIG, HIVE (Query Data): HDInsight, Azure Synapse Analytics, Amazon EMR, Google Cloud Dataproc

    🔹6. HBase (NoSQL DB): Azure Cosmos DB, HBase on a virtual machine (VM), HBase in Azure HDInsight, Amazon DynamoDB, Google Cloud Bigtable

    🔹7. Mahout, Spark MLLib (ML Libraries): Databricks, Amazon SageMaker, No direct equivalent in GCP

    🔹8. Solar, Lucene (Search/Index): Azure Cognitive Search, Amazon CloudSearch, Google Cloud Search

    🔹9. Zookeeper (Cluster Management): No direct equivalent in Azure, Amazon Managed Apache ZooKeeper, No direct equivalent in GCP

    🔹10. Oozie (Job Scheduling): Azure Data Factory, AWS Step Functions, Google Cloud Composer

  22. Software defined Networking(SDN) Northbound vs Southbound

    Expand 
    graph TD
    A[Application layer - routing, load balancing, etc] -->|Northbound APIs| B[Control layer - SDN controller]
    B -->|Southbound APIs| C[Infrastructure layer - physical switches, data plane]
    Loading

    🔹The Controller is the SDN network's brain, directing traffic flows.

    🔹 The Southbound Interface communicates the controller's decisions to the switches using protocols like OpenFlow.

    🔹SDN Switches direct traffic based on the controller's instructions.

    🔹Network Devices (servers, routers, etc.) send and receive data flows as directed by the SDN switches.

    🔹The Northbound Interface uses APIs to exchange data between the controller and applications.

    🔹SDN Applications use network data to perform tasks, communicating their needs to the controller.

    graph LR
A[Controller] -- API --> B[Southbound Interface]
B -- OpenFlow --> C[SDN Switches]
C -- Data Flow --> D[Network Devices]
A -- API --> E[Northbound Interface]
E -- Applications --> F[SDN Applications]
    Loading

  23. Cracking coding interviews

    Expand

    🔹src: ref

    🔹Two Pointers: Navigating arrays with two indices. ref

    🔹Intervals: Working with ranges of values. ref / ref / ref

    🔹Dynamic Programming: Solving complex problems by breaking them down into simpler subproblems. ref / ref

    🔹Tree Traversal: Visiting all nodes in a tree. ref / ref

    🔹DFS-BFS: Depth-first and breadth-first search algorithms. ref / ref / ref / ref

    🔹Binary Search: Finding an element in a sorted array. ref

    🔹Array: A data structure holding elements. ref

    🔹Sliding Window: A subset of data that moves. ref / ref / ref / ref / ref

    🔹Backtracking: Trying out all possibilities to find a solution. ref / ref / ref

    🔹Combination: Finding all possible arrangements of elements. ref

    🔹Trie: A tree-like data structure for storing strings. ref

    🔹Word Break: Dividing a string into words. ref

    🔹Bit Manipulation: Performing operations on binary numbers. ref / ref

    🔹Sum: Adding numbers together. ref

    🔹Monotonic Stack: A stack keeping elements in an ordered manner. ref

    🔹Big-O-Notation: ref

    🔹src: ref

    🔹Master Graph Algorithms for Coding Interviews

    🔹20 Patterns to Master Dynamic Programming

    🔹LeetCode was HARD until I Learned these 15 Patterns

    🔹How I Mastered Data Structures and Algorithms

  24. Medallion architecture: A data design pattern for lakehouses. It enhances data quality across three layers: bronze (raw), silver (curated), and gold (presentation). This “multi-hop” architecture allows data to transition between layers as required. ref

  25. Slowly changing dimensions (SCD): Slowly Changing Dimensions change over time, but at a slow pace and unpredictably. For example, a customer’s address in a retail business.

  26. Star schema: The Star Schema is a data model for data warehouses. It has a central fact table for measurable data and surrounding dimension tables for descriptive data. ref

  27. OLAP vs OLTP: OLAP: Used for complex data analysis and business reporting, such as financial analysis and sales forecasting. OLTP: Used for real-time processing of online transactions, including everyday transactions like ATM withdrawals and in-store purchases.

  28. Test-Driven Development

    Expand

    🔹F.I.R.S.T: A testing principle where tests are Fast, Isolated, Repeatable, Self-validating, and Timely/Thorough.

    🔹DAMP: Stands for "Descriptive And Meaningful Phrases" in testing.

    🔹BDD: Behavior Driven Development uses "Given-When-Then" format.

    🔹DRY: "Don’t Repeat Yourself" principle avoids redundancy.

    🔹TDD: Test-Driven Development focuses on tests first.

    🔹Exploratory Testing: Simultaneous learning, test design, and test execution. It is about exploring the application and finding defects that were not anticipated.

    🔹Smoke Testing: A preliminary test to check the basic functionality of an application to ensure that the most crucial functions work.

    🔹Alpha/Beta Testing: 🔹Alpha: Initial testing performed by internal staff / 🔹Beta: Testing performed by actual users

  29. SLA, SLO, and SLI

    Expand

    🔹SLA (Service Level Agreement): A contract defining the expected level of service. 99.9% uptime

    🔹SLO (Service Level Objective): A measure of service performance agreed upon in an SLA. 200ms response

    🔹SLI (Service Level Indicator): A quantitative measure of a specific aspect of the level of service. Query latency

  30. Space-Based Architecture (SBA) vs Cell-based architecture

    • Space-Based Architecture (SBA): Removing the database and instead using a shared memory (memory grids) model
    • Cell-based architecture: multiple isolated workload instances (cells) for fault isolation and handling subsets of workload requests

  31. Azure vs AWS vs GCP in Cloud Netwrok

    Expand

    🔹Cloud Network Structures:

    • AWS: region → VPC → availability zone → subnet (public, private)
    • Azure: region → VNet → availability zone → subnet
    • GCP: global → VPC → subnet (region-specific)

    🔹Traffic Between VNet or VPC:

    • Set up a VNet or VPC gateway and configure the routing table.
    • Azure: One VPN gateway per virtual network, supporting external and on-premise connectivity.
    • Azure: Routing tables can be assigned at the subnet level.
    • AWS: Routing tables are associated at the VPC level, determining traffic between subnets.
    • GCP: Routing tables are associated with the VPC, not the subnet. Routes are determined by subnet CIDR ranges.

    🔹Regional Traffic:

    • Use peering for traffic between VNets or VPCs in the same region.
    • Peering provides lower latency, higher bandwidth, and reduced costs compared to gateways

    🔹Hybrid Connectivity:

    • Required for on-premise networks to connect to cloud networks.
    • Services used: Azure ExpressRoute, AWS Direct Connect, GCP Cloud Interconnect.

    🔹Connectivity Scenario

    Connectivity Scenario AWS Azure GCP
    VPC (VNet) <-> VPC (VNet) VPC Peering, AWS Transit Gateway VNet Peering, Azure Virtual WAN, VNet Gateway VPC Network Peering, Cloud VPN
    On-Premise <-> VPC (VNet) Virtual Private Gateway, AWS Direct Connect, AWS Transit Gateway VPN Gateway, Azure ExpressRoute, Azure Virtual WAN Cloud VPN, Cloud Interconnect
    VNet (VPC) <-> Internet Internet Gateway (public subnet), NAT Gateway (private subnet) Internet Gateway, NAT Gateway Cloud Router, Cloud NAT
    Subnet Traffic Control Network Access Control Lists (ACLs) Network Security Groups (NSGs) 1. Shared VPC (XPN) subnets and set permissions at project or subnet level 2. Firewall Rules (VM level).
    • In Azure, the equivalent of AWS Transit Gateway is Azure Virtual WAN
    • Azure and GCP don’t have a direct concept of public and private subnets like AWS.
    • An Azure Virtual Network (VNet) Gateway can serve as a VPN gateway or an ExpressRoute gateway. Site-to-Site, Point-to-Site, and VNet-to-VNet connections all use a VPN gateway.
      • Point-to-site: Individual device to network.
      • Site-to-site: Network to network.
    • Private IP vs Public IP: 192.168.1.4 (not routable on the internet) vs 34.207.152.137

    🔹Resource hierarchy

    Level AWS Azure GCP
    1 Organization Management Group Organization
    2 Organizational Unit (OU) Subscription Folder
    3 Account Resource Group Project
    4 Resources Resources Resources

  32. Security Words 101

    Expand

    • Identity and Access Management

      • MIM/PAM: Microsoft Identity Manager / Privileged Access Management
      • PAW: Privileged Account Workstations
      • AADIS: Azure Active Directory Implementation Services
      • DIAD: Design and Implementation for Azure Active Directory
      • LAPS/SLAM: Local Administrator Password Solution / Security Lifecycle Automation & Management
      • IAM: Identity and Access Management
      • SSO: Single Sign-On
      • MFA: Multi-Factor Authentication

    • Threat Detection and Response

      • ATA: Advanced Threat Analytics
      • PADS: Persistent Advisory Detection Service
      • IR&R: Incident Response & Recovery
      • ATP: Advanced Threat Protection
      • OMS: Operations Management Suite
      • ETD: Enterprise Threat Detection
      • SIEM: Security Information and Event Management
      • EDR: Endpoint Detection and Response
      • XDR: Extended Detection and Response
      • SCEP: System Center Endpoint Protection (Microsoft Defender for Endpoint)

    • Information Protection

      • AIP: Azure Information Protection (=AD RMS+On-premise files)
      • AD RMS: Active Directory Rights Management Services
      • WIP: Windows Information Protection
      • DLP: Data Loss Prevention
      • IRM: Information Rights Management

    • Security Development and Assessment

      • SDL: Security Development Lifecycle
      • MSRA: Microsoft Security Risk Assessment
      • DIF: Dynamic Identity Framework
      • OAWSS: Offline Assessment for Windows Server Security
      • OAADS: Offline Assessment for Active Directory Security
      • SAST: Static Application Security Testing
      • DAST: Dynamic Application Security Testing

    • Security Management

      • ESAE: Enhanced Security Administrative Environment
      • SCCM: System Center Configuration Manager
      • EMS: Enterprise Mobility Suite
      • SCOM/ACS: System Center Operations Manager / Audit Collection Services
      • GRC: Governance, Risk, and Compliance
      • SOC: Security Operations Center

  33. Cloud Design Patterns. ref

    Expand

    Cloud Design Patterns Summary

    1. Data Management 📊

    • Cache-Aside: Cache frequently used data for performance.
    • CQRS: Separate reads/writes for scalability.
    • Event Sourcing: Record full data change history.
    • Materialized View: Precompute query results for speed.
    • Sharding: Partition data to scale storage.

    2. Design and Implementation 🛠️

    • Strangler Fig: Gradually migrate legacy systems.
    • Anti-Corruption Layer: Isolate new systems from old ones.
    • Bulkhead: Prevent failure spread across components.
    • Sidecar: Add functionality without changing the core.
    • BFF: Tailor backend for different clients.

    3. Messaging 📨

    • Queue-Based Load Leveling: Buffer requests for smooth load handling.
    • Publisher-Subscriber: Broadcast messages to multiple consumers.
    • Competing Consumers: Process messages in parallel for scalability.
    • Message Broker: Route messages via intermediary.
    • Pipes and Filters: Sequentially process data through components.

    4. Security 🔒

    • Valet Key: Provide secure temporary access.
    • Gatekeeper: Filter requests to protect backends.
    • Federated Identity: Use third-party login credentials.
    • Secret Store: Secure sensitive data.
    • Validation: Ensure data input is sanitized.

    5. Reliability ⚙️

    • Retry: Retry failed operations automatically.
    • Circuit Breaker: Stop repeated failing operations.
    • Throttling: Limit resource usage via request control.
    • Health Endpoint Monitoring: Expose health checks for monitoring.

  34. Principles & Concepts: YAGNI, KISS, DRY, CAP Theorem, PACELC, ACID, BASE

    Expand
    • YAGNI (You Aren't Gonna Need It): Don’t add features until necessary.
    • KISS (Keep It Simple, Stupid): Keep designs simple.
    • DRY (Don't Repeat Yourself): Avoid code duplication.
    • CAP Theorem (Consistency, Availability, Partition Tolerance): Choose between consistency, availability, and partition tolerance.
    • PACELC (Partition Tolerance, Availability, Consistency, Else Latency/Consistency): Trade-offs exist in availability/consistency and latency/consistency.
    • ACID (Atomicity, Consistency, Isolation, Durability): Properties ensuring reliable database transactions.
    • BASE (Basically Available, Soft State, Eventually Consistent): Prioritizes availability and eventual consistency in distributed systems.

  35. Windows UI Development Frameworks: WinUI3, Windows Form, WPF, UWP, Win32, .NET MAUI, Uno

    Expand
    • For new Windows apps, use the Windows App SDK and WinUI instead of UWP, which is no longer actively developed.
    • .NET MAUI is backed by Microsoft, whereas the Uno Platform is supported by nventive.
    graph TD
    A[WinUI 3 <br/>'Windows UI Library 3 <br/>Supports only Windows'] -->|Builds on| B[UWP <br/>'Universal Windows Platform <br/>Supports only Windows']
    A -->|Supports| E[.NET MAUI <br/>'Multi-platform App UI <br/>Cross-platform support: not support Linux']
    B -->|Shares components with| E
    C[WPF <br/>'Windows Presentation Foundation <br/>Supports only Windows'] -->|Integrates with| E
    D[Windows Forms <br/>'Older framework <br/>Supports only Windows'] -->|Legacy framework| C
    F[Uno Platform <br/>'Cross-platform apps <br/>Cross-platform support incl. WebAssembly'] -->|Uses| A
    F -->|Leverages| B
    G[Win32 MFC <br/>'Microsoft Foundation Classes <br/>Supports only Windows'] -->|Legacy framework| C
    H[PWA <br/>'Progressive Web Apps <br/>Cross-platform support'] -->|Builds on| B
    I[React Native for Windows <br/>'Cross-platform mobile framework <br/>Cross-platform support'] -->|Supports| A
    J[Blazor Hybrid <br/>'Web UI with native capabilities <br/>Cross-platform support'] -->|Integrates with| E

    %% Set styles for the backgrounds
    style A fill:#f3e5f5,stroke:#6a1b9a,stroke-width:2px;  %% WinUI 3 as Windows-only
    style B fill:#f3e5f5,stroke:#6a1b9a,stroke-width:2px;  %% UWP as Windows-only
    style E fill:#e0f7fa,stroke:#4d94ff,stroke-width:2px;  %% Cross-platform
    style F fill:#e0f7fa,stroke:#4d94ff,stroke-width:2px;  %% Cross-platform
    
    style C fill:#f3e5f5,stroke:#6a1b9a,stroke-width:2px;  %% WPF as Windows-only
    style D fill:#f3e5f5,stroke:#6a1b9a,stroke-width:2px;  %% Windows Forms as Windows-only
    style G fill:#f3e5f5,stroke:#6a1b9a,stroke-width:2px;  %% Win32 MFC as Windows-only

    %% Set styles for web app frameworks
    style H fill:#c8e6c9,stroke:#388e3c,stroke-width:2px;  %% PWA as Cross-platform
    style I fill:#c8e6c9,stroke:#388e3c,stroke-width:2px;  %% React Native for Windows
    style J fill:#c8e6c9,stroke:#388e3c,stroke-width:2px;  %% Blazor Hybrid
    
    %% Add color legend
    K[Legend: <br/> <span style='color:#4d94ff'>Blue Background:</span> Cross-platform support <br/> <span style='color:#6a1b9a'>Purple Background:</span> Supports only Windows <br/> <span style='color:#388e3c'>Green Background:</span> Web app frameworks] 
    
    %% Set styles for the legend background
    style K fill:#fff9c4,stroke:#f57f17,stroke-width:2px;

    %% Comments
    click A "https://docs.microsoft.com/en-us/windows/apps/winui/winui3/" "WinUI 3: Modern UI framework for Windows apps."
    click B "https://docs.microsoft.com/en-us/windows/uwp/" "UWP: Build universal apps for all Windows devices."
    click C "https://docs.microsoft.com/en-us/dotnet/desktop/wpf/" "WPF: Rich desktop applications with advanced graphics."
    click D "https://docs.microsoft.com/en-us/dotnet/desktop/winforms/" "Windows Forms: Simplified desktop app development."
    click E "https://docs.microsoft.com/en-us/dotnet/maui/" ".NET MAUI: Cross-platform UI framework for mobile and desktop."
    click F "https://platform.uno/" "Uno Platform: Build cross-platform apps using WinUI."
    click G "https://learn.microsoft.com/en-us/cpp/mfc/overview-of-mfc?view=msvc-160" "Win32 MFC: C++ framework for Windows desktop applications."
    click H "https://learn.microsoft.com/en-us/microsoft-edge/progressive-web-apps-chromium/" "PWA: Build web apps that behave like native apps."
    click I "https://microsoft.github.io/react-native-windows/" "React Native for Windows: Build cross-platform apps using React Native."
    click J "https://docs.microsoft.com/en-us/aspnet/core/client-side/blazor/hybrid?view=aspnetcore-7.0" "Blazor Hybrid: Build web UIs with native capabilities."
    Loading

  36. Mixin = Interface + actual implementation. It is a class that provides reusable methods/behavior to other classes without requiring full inheritance.