-
Notifications
You must be signed in to change notification settings - Fork 1
/
Neuromorphic-Thermogenesis.txt
1535 lines (1219 loc) ยท 162 KB
/
Neuromorphic-Thermogenesis.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Neuromorphic Thermogenesis
ChatGPT 4
User
Category Theory is a field of study that emerged around 1940s. By that time, the tangled web of inquiries in mathematics had accumulated many ideas from other fields. The inventors used Category Theory as a ground for unifying these different ideas under a common set of abstractions by studying these broad ranging ideas deeply. It was introduced by Samuel Eilenberg and Saunders Mac Lane to study algebraic topology, where they wanted to introduce the idea of natural transformation to make tractable the idea of naturality occuring on functors between functors (A functor is a category theoretical term that roughly means functions and the idea of naturality can very roughly be compared to the idea of talking about a concept without a given frame of reference).
If you enjoy reading the history of the subject before diving into it, I highly recommend this accessible work by Garlandus describing the emergence of Category Theory. Garlandusโ essay holds a high place in my mind, it is the kind of work that gives a sufficient background on the field discussing details of how it took shape as a result of contributions between multiple people. He locates Hilbertโs Gรถttingen as a central locus that lead to the germination of work on Category Theory.
Now, seeing that the subject emerged from algebraic topology, the application to Computer Science may seem arbitrary. But this came after deep investigations into computational structures and programming since around 1960s, where multiple links between algebra, automata, computational control/data structures, recursion schemes were unearthed. If you are interested in knowing about it more deeply, I have a repo, which is a work in progress, where I try to unravel these links. A curious fact that I have to mention is that, one of the co-inventors of Category Theory, Samuel Eilenberg, went on to apply his ideas to automata after inventing Category Theory!
A nice recent article is penned by Emily Riehl showing how โ-categories act as a great foundation for mathematics up here. It is written in an accessible manner to give a birdโs eye view of the whole field showing how it interconnects with other mathematical fields.
Now that we have seen how Category Theory came around, let us look at the resources that can guide us in building a working knowledge of the subject.
ChatGPT
It's fascinating to see how Category Theory, originally formulated to address issues in algebraic topology, has grown to influence various fields, including computer science. This expansion underscores the versatility and depth of mathematical concepts and their applicability beyond their initial contexts.
Essential Resources for Learning Category Theory
Introductory Texts
"Concepts of Category Theory" by Haskell Curry: This book offers a gentle introduction to the concepts of categories, functors, and natural transformations, making it ideal for those new to the field.
"Category Theory for Programmers" by Bartosz Milewski: Tailored for programmers, this book bridges the gap between abstract mathematical concepts and practical software development, demonstrating the utility of category theory in functional programming and computer science.
Advanced Studies
"Higher Category Theory" by Jacob Lurie: For those interested in diving deeper, this book explores higher-dimensional categories which are crucial in modern theoretical physics and complex mathematical structures.
"Sheaves in Geometry and Logic" by Saunders Mac Lane and Ieke Moerdijk: This text provides an in-depth look at how category theory is applied in areas like geometry and logic through the concept of sheaves.
Online Resources
nLab: A comprehensive wiki that includes extensive articles on category theory and its applications across mathematics and physics.
Category Theory Course by Emily Riehl: Available on YouTube, this series of lectures offers an accessible yet thorough introduction to the subject, based on her book "Category Theory in Context."
Workshops and Conferences
Participating in workshops and conferences can provide insights from leading experts in the field and opportunities for collaborative learning and research. Websites like MathMeetings.net often list upcoming events in category theory and related areas.
Community and Forums
Stack Exchange (Mathematics and Theoretical Computer Science sections): These forums are valuable for both beginners and experts to discuss concepts and problems in category theory.
Reddit communities such as r/math and r/compsci: These platforms allow for informal discussions and sharing of resources about the theoretical and practical aspects of category theory.
By exploring these resources, you can build a solid foundation in Category Theory and appreciate its broad applications across different areas of mathematics and technology. Whether you're a mathematician, a computer scientist, or simply a curious learner, Category Theory offers rich insights and tools for understanding complex systems and abstractions.
User
Because of the standing of Category Theory as a general domain of inquiry, I have constantly felt it better to first get a cursory knowledge in functional programming and abstract algebra before tackling the subject. These paradigms are closer to the work done in Category Theory. If you come around to Category Theory without having some concrete experience in dealing with problems to which you can relate the ideas you learn here, I feel it might be a bit hard to connect with this domain of study and gain intuition into to why such machinery is needed.
Functional programming techniques are a way in which a lot of these concepts find a space to talk about them and ground them in a concrete setting where their behaviours can be probed and pondered. So I proceed to recommend some resource that might help you develop some functional programming knowledge before you come around to Category Theory.
How to Design Programs (First Edition)
How to Design Programs is an accessible introduction to functional programming. The book is thorough in its coverage of essential ideas and slowly walks a novice through exercises to understand the concepts and how to put them to use in practical problems that one may encounter in their daily practice.
Introduction to Functional Programming
Introduction to Functional Programming by Richard Bird and Philipp Wadler is a more formal treatment of functional programming. They use the language Miranda to elucidate the concepts.
Structure and Interpretation of Computer Programs
SICP by Abelson and Sussman is considered a classic text that provides a synthesis of great ideas of computer science using LISP. It is a natural second choice once you have mastered the ideas in HtDP.
Reading some or all of the books here will give you a solid footing in functional programming. With that knowledge under your belt, next step is to develop sufficient understanding of the field of mathematics. This knowledge would make the transition into CT much more smooth as there are deep links within CT that lead you onto very different domains and having a well-rounded knowledge of the field would help you to ease into fields which might be remote from your experience, but would nevertheless hold the gold, so to speak.
Breadth and Depth
Mathematics Form and Function
To make this case, let me first mention this book by Saunders Mac Lane, the co-inventor of Category Theory, where he gives a tour of ideas of mathematics in a comprehensive way. This book doubles out as an atlas of mathematical inquiry as there are multiple maps in this book that give you an idea on how to navigate the field. Such is the kind of breadth and depth that has gone into building Category Theory. I donโt mean this as something to intimidate the reader, but just to provide the awareness that the field is vast and finds application in many areas, computation being one among many. Mastery of all these fields is not a pre-requisite to be proficient in it. But it has to be pointed out that the masters of the subject have done a great deal of exploration to bring forth the fruits of Category Theory.
Introduction through Abstract Algebra
Now an adjacent field to Category Theory is Abstract Algebra. The route I have taken is to first learn abstract algebra which undergirds the algebraic species that are often subjects of study in Category Theory. To make sense of these, let me mention a few books. It is not mandatory reading, if you want to dive first into Category Theory. In fact some of the books that I have put together here allows one to learn Category Theory without much prerequisite knowledge but abstract algebra is a field that I have felt is most proximate and has aided me ease into Category Theory texts.
A Book of Abstract Algebra
Charles Pinter
This book by Charles Pinter reads not like a textbook but like a description of the field.
A Concrete Approach to Abstract Algebra
W. W. Sawyer
A narrative approach for Abstract Algebra is given by Sawyer by focussing on concrete applications.
Visual Group Theory
Nathan Carter
Visual Group Theory by Nathan Carter gives an overview of the field by relying on visualizations. His playground for the cayley diagram explorations gives a good idea of the quality of work that has gone into producing this book.
I will try to expand on this catalogue once I have better perspective, but these three seem to be the most promising to get a good overview of the algebraic structures.
Once the above works are studied, starting with Category Theory texts should be an easy process. I will now proceed to list the works which will help an enthusiast to navigate the field of Category Theory and understand it deeper.
ChatGPT
You've outlined an excellent approach to learning Category Theory by building a strong foundation in related fields like functional programming and abstract algebra. This method not only enriches understanding but also provides practical contexts in which these abstract concepts can be applied. Let's now delve into the resources that can help bridge these foundational topics to Category Theory itself.
Functional Programming
Before diving into Category Theory, sharpening skills in functional programming can be tremendously beneficial. Here are some foundational books you mentioned and why they're valuable:
"How to Design Programs" (First Edition): This book is perfect for beginners and thoroughly covers fundamental concepts, leading readers through practical exercises that enhance their problem-solving skills in a functional paradigm.
"Introduction to Functional Programming" by Richard Bird and Philipp Wadler: Offers a more formal approach to functional programming, using Mirandaโa language designed to teach the principles of functional programming without the overhead of more complex languages.
"Structure and Interpretation of Computer Programs" (SICP) by Abelson and Sussman: Considered a seminal text in computer science education, SICP uses Scheme, a dialect of Lisp, to explore significant programming concepts. It's ideal after gaining some initial grounding in functional programming.
Abstract Algebra
Equipping yourself with knowledge in abstract algebra can provide a deeper understanding of the structures often explored in Category Theory:
"A Book of Abstract Algebra" by Charles Pinter: This book offers an accessible introduction to abstract algebra with clear explanations, making it suitable for beginners.
"A Concrete Approach to Abstract Algebra" by W. W. Sawyer: Focuses on the applications of abstract algebra, providing a more tangible understanding of its concepts.
"Visual Group Theory" by Nathan Carter: Uses visualizations to explain group theory, an essential part of abstract algebra, making complex topics more approachable.
Transition to Category Theory
After building a background in these areas, transitioning to Category Theory will be more intuitive. Here are some texts to help you navigate and understand Category Theory deeper:
"Category Theory for the Sciences" by David Spivak: An excellent resource for those from different scientific backgrounds, providing applications of category theory in various fields.
"An Introduction to Category Theory" by Harold Simmons: This book offers a straightforward approach to the core concepts of category theory, suitable for those who have some background in mathematics.
"Category Theory" by Steve Awodey: Aimed at advanced undergraduates and beginning graduate students, this text introduces category theory at a higher level.
"Basic Category Theory" by Tom Leinster: This is another excellent introductory text, which focuses on the basic concepts and is known for its clarity and accessibility.
By following this structured approachโstarting from functional programming, moving through abstract algebra, and finally diving into category theoryโyouโll not only gain a comprehensive understanding of each field but also see how they interconnect and reinforce each other in various applications. This holistic view is essential for grasping the profound implications of category theory in mathematics and beyond.
User
Visual Nature of Category Theory
Category Theory is the study of objects and morphisms and for this purpose, I find it most important to have a visual setting for exploring these ideas. Many of the ideas being talked about in Category Theory spawns dynamic pictures of morphisms in my head, but I find it hard to visualize them as there is so little Category Theory with pictures around. What is pictured here is an animation by James McKeown of a modular lattice rotating on its vertical axis. These sort of algebraic structures are a part of what we study with Category Theory. And I think there is a certain truth to the idea that geometry is the missing link to ground the abstract ideas that is being studied under Category Theory.
In the orientation section of the paper by Lawvere on Grassmanโs mathematical work there is a really nice paragraph on the significance of category theory and how it functions as an instrument of analysis and dialectics by serving as a โuniversal geometric calculusโ:
Grassman in his philosophical introduction describes the two-fold division of formal sciences, that is, the science of thinking, into dialectics and mathematics. He briefly describes dialectics as seeking the unity in all things, and he describes mathematics as the art and practice of taking each thought in its particularity and pursuing it to the end. There is a need for an instrument which will guide students to follow in a unified way both of these activities, passing from the general to the particular and from the particular to the general.
Image showing how mathematics takes the particular to the general and dialectics takes the general to the particular
I believe that the theory of mathematical categories (which was made explicit 50 years ago by Eilenberg and Mac Lane, codifying extensive work done by Hurewicz in particular during the 1930โs), can serve as such an instrument. It was introduced and designed in response to a very particular question involving passage to the limit in calculating cohomology of certain portions of spheres, but this particular calculation necessitated an explicit recognition of the manner in which these spaces were related to all other spaces and, in particular, how their motion might induce other motions. In other words, category theory was introduced (and still serves) as a โuniversal geometrical calculusโ.
ChatGPT
The visualization of concepts in Category Theory can indeed significantly enhance understanding, particularly given the abstract nature of the subject. As you noted, the use of visual aids like animations of algebraic structures can provide a tangible sense of how these theories operate and interconnect. This approach aligns well with Hermann Grassmann's views on mathematics as a discipline that bridges particularity with generality, and the role of dialectics in exploring the unity in diversity.
Visualizing Category Theory
Category Theory, while abstract, lends itself well to geometric and visual interpretations, as the relationships between objects (via morphisms) often have spatial analogies that can be represented visually. Here are some ideas and resources that might help in visualizing these concepts more effectively:
Animated Diagrams and Simulations: Visual representations, like the one you mentioned by James McKeown, can be effective in showing how different structures within Category Theory interact and transform. Software tools such as GeoGebra, Mathematica, and even Python libraries like Matplotlib or Plotly, can be used to create animations that help in visualizing these structures.
Interactive Platforms: Tools like Algebrarules.com, which visually and interactively demonstrates algebraic identities and properties, could inspire similar platforms for Category Theory. Interactive web-based tools can allow users to manipulate objects and morphisms to see how different compositions and structures evolve.
Graphical Abstracts and Sketches: Books and papers on Category Theory could include more graphical abstracts that depict the flow of logic and relationships in a visual manner. Sketches can help in breaking down complex ideas into more digestible parts, making it easier to understand the flow from one concept to another.
Educational Videos: Channels on platforms like YouTube can offer visual tutorials that break down Category Theory concepts into simpler, understandable parts using diagrams and on-screen animations. This approach can be particularly effective for auditory and visual learners.
Incorporating Geometry into Category Theory
The concept of a "universal geometric calculus" that you referenced from Lawvere's paper highlights the potential of using geometric insights to ground the abstract notions of Category Theory. This idea also ties back to Grassmann's division of the formal sciences and his philosophical stance on the unity and particularity of thoughts:
Geometric Category Theory: There could be a subfield or a series of studies dedicated to exploring Category Theory through geometric lenses, focusing on how geometric intuition can be used to understand and solve categorical problems.
Courses and Workshops: Academic courses and workshops could include modules that specifically focus on the visualization and geometric interpretations of categorical concepts, providing tools and software training for creating these visualizations.
Collaborative Research: Encouraging collaborations between mathematicians, computer scientists, and visual artists could lead to new ways of representing and understanding Category Theory. This could involve developing new software tools or artistic representations that make the field more accessible and visually engaging.
By enhancing the visual and geometric aspects of Category Theory, the field could become more intuitive and approachable, particularly for those new to its concepts or from other disciplines where visual and spatial reasoning is more common.
User
Some notes on Category Theory
As a unifying field
Category theory becomes a setting from which you can understand the connection between various mathematical objects that come couched in the contingent scaffolding of a particular theory. Category Theory allows one to map out the structure preserving mappings of a theoryโs connection or morphism to other theoretical settings. These mappings go by the name of various โmorphismsโ, common ones being homomorphism and isomorphism i.e. partial/total equivalent representations of a source representation to a target representation. By adopting such an approach, Category Theory acts as a tool of knowlege transfer and as a medium that helps transfer conceptual knowledge among disparate domains in mathematics.
TODO: List some domains which are far removed at a cursory glance, but is brought together with Category Theory.
A relational setting
Set theory is seen as a canonical extensional setting where one-to-one correspondence between different mathematical objects is established. Category Theory shifts this focus from elements of the set or any particular theory and asks about what kind of connections exist between two different mathematical structures. This approach gains more patency as we move towards an algebraic approach where mathematical objects are determined and understood by the network of relationships they enjoy with other objects. The idea when operating in Category Theory is that the structure of a mathematical object is sought in the network of relationships it enjoys with other structures of the same kind. These connections are characterized by the morphisms and natural transformations the structure partakes in. A mathematical object in Category Theory gets defined as the sum total of network of relations that object is involved with others. In some sense, this metamathematical outlook seems to derive from topology where connection between two spaces are talked about and how one is the other but modulo some property.
TODO: Elaborate and integrate this idea into this prose. I canโt help but feel that this is a crystallization of the idea of manifolds as devised by Bernhard Riemann polished to a fine theoretical setting.
Since a lot of modern mathematics since 1900s was developed using set theory, Category theory assimilates these results by talking about the category of sets. But it is not limited to this vantage point. Categories provide a way to interpret concepts within a category relationally without the need for set theory to back it up.
There is also a way in which Category Theory shows affinity with Synthetic Differential Geometry / Smooth Infinitesimal analysis thereby providing a setting in which continuous concepts can be grounded using this theory.
TODO: Learn more about the relationship between sets and category theory and present a concrete instance where they display pros/cons.
Elements of Category Theory
Category Theory has nouns (objects), verbs (morphisms), associative law, and composition. Using these basic properties as the primitives, it generates templates with which we can package mathematical objects of arbitrary complexities. Objects of a category are the instances of a certain associated form and transformation between these instances are โmorphismsโ which preserve the structure of these instances in some specified way.
Category Theory and Linguistics
With Category Theory, we are moving into a setting where the duality between nouns and verbs find a rigorous expression. That being the case, it might come as no surprise that there is something called Categorical logic which attempts to ground linguistic phenomena and natural language grammar in a category theoretical setting.
TODO: Briefly sketch the work of people using Category Theory for linguistics. These might be helpful:
The Development of Categorical Logic
The History of Categorical Logic 1963-1977
ChatGPT
Your notes on Category Theory offer a deep dive into its essence as both a mathematical and conceptual framework. Let's elaborate and integrate some of these ideas to provide a clearer, more comprehensive view.
Category Theory as a Unifying Field
Category Theory excels as a unifying field, making it possible to discern connections across various mathematical domains that might seem unrelated at first glance. For instance, the fields of topology, algebra, and logic can all be approached through the lens of Category Theory, revealing deep structural similarities and differences:
Topology and Algebra: Category Theory shows how structures like topological spaces and algebraic groups can be linked through continuous or structure-preserving functions (morphisms). An example is the fundamental group in topology, which can be studied using concepts from group theory.
Logic and Set Theory: In logic, categories can encapsulate relationships between different logical systems, showing how propositions and proofs from one system can be mapped to another, preserving logical structure through functors.
A Relational Setting: From Set Theory to Categories
Your discussion rightly shifts the focus from elements to relationships, which is a core principle of Category Theory. This relational view is indeed reminiscent of topology, where the concept of a continuous map (a morphism in categorical terms) is more significant than the properties of individual points. This shift is also akin to the idea of Riemannian manifolds in geometry, where the local properties of spaces are understood in relation to their global structure.
Category Theory provides a framework where an object is defined not just by its inherent properties but by its relationships with other objectsโthis is analogous to defining a space by the set of maps that come into it or go out of it (its homotopy or homology groups in topology).
Category Theory and Set Theory
Discussing the interplay between Category Theory and Set Theory can highlight how categories provide a more flexible framework for dealing with mathematical structures, especially when traditional set-theoretic methods are cumbersome or inadequate. For example, in dealing with large or complex structures like classes of all classes, Category Theory offers tools like Grothendieck universes to handle these issues without the paradoxes often encountered in naive set theory.
Category Theory in Linguistics
The use of Category Theory in linguistics is particularly fascinating as it applies mathematical structure to language analysis. Categorical logic, as you've mentioned, helps model how sentences are constructed in natural languages, transforming the abstract structures of grammar into concrete mathematical frameworks. Researchers like Joachim Lambek have applied categorical structures to model syntactic and semantic compositions, showing that grammatical constructions in languages follow patterns that can be rigorously described by functors and natural transformations between categories.
Potential Topics to Expand Upon
Detailed Examples in Topology and Algebra: Discuss specific cases like the use of category theory in understanding the properties of covering spaces or group actions on sets.
Categorical Logic in Linguistics: Provide a more detailed examination of how categorical logic has been used to model linguistic structures and what insights it has brought to understanding grammar and syntax.
Synthetic Differential Geometry and Category Theory: Explore this relationship further, especially how Category Theory supports frameworks in which differentiability and continuity can be handled in algebraic terms, offering new insights into the calculus and its foundations.
By expanding on these points, your notes will not only deepen the understanding of Category Theoryโs broad applications but also highlight its powerful unifying capabilities across diverse mathematical and scientific disciplines.
User
That was from https://github.com/prathyvsh/category-theory-resources. Could I get a summary?
ChatGPT
Sure, I can provide a summary based on your previous message which seems to be an excerpt from a larger piece of writing on Category Theory.
Summary of Category Theory Notes
1. Category Theory as a Unifying Field:
Main Idea: Category Theory serves as a powerful unifying framework in mathematics, allowing for the mapping and understanding of relationships between various mathematical structures across different domains. It emphasizes morphisms (like homomorphisms and isomorphisms) which are mappings that preserve structure from one set to another, facilitating knowledge transfer between disparate mathematical domains.
2. Category Theory in a Relational Context:
Main Idea: Unlike set theory, which focuses on elements and their properties, Category Theory shifts the emphasis to the relationships between different mathematical structures. This approach is more algebraic and focuses on the network of relationships, defining mathematical objects by their interactions with others. This perspective is influenced by concepts from topology, where the focus is on the properties of spaces in relation to one another.
3. Interplay Between Category Theory and Set Theory:
Main Idea: While Category Theory originated within the framework of set theory, it extends beyond to provide more general and flexible tools for dealing with mathematical structures. This is particularly useful in areas like the theory of large cardinals or complex class structures, where traditional set theory might encounter logical paradoxes.
4. Application in Linguistics:
Main Idea: Category Theory also finds applications outside mathematics, such as in linguistics, where it helps model grammatical structures and sentence construction using categorical logic. This approach demonstrates the adaptability of Category Theory to analyze and structure information in fields beyond pure mathematics.
5. Suggested Expansion Topics:
Potential Areas for Further Exploration:
Detailed examples of Category Theory applications in topology and algebra.
Exploration of categorical logic in linguistic models.
Examination of the relationship between Category Theory and synthetic differential geometry.
This summary reflects the core themes of Category Theory as discussed in your notes, emphasizing its role as a unifying, relational, and interdisciplinary tool in both mathematics and beyond.
User
https://www.scientificamerican.com/article/infinity-category-theory-offers-a-birds-eye-view-of-mathematics1/
OCTOBER 1, 2021
20 MIN READ
Infinity Category Theory Offers a Birdโs-Eye View of Mathematics
Mathematicians have expanded category theory into infinite dimensions, revealing new connections among mathematical concepts
BY EMILY RIEHL
Infinity art concept composed of square blocks.
Eric Petersen
October 2021 Issue
Mathematics
On a crisp fall New England day during my junior year of college, I was walking past a subway entrance when a math problem caught my eye. A man was standing near a few brainteasers he had scribbled on the wall, one of which asked for the construction, with an imaginary straightedge and compass, of a cube with a volume twice that of a different, given cube.
This stopped me in my tracks. I had seen this problem before. In fact, the challenge is more than two millennia old, attributed to Plato by way of Plutarch. A straightedge can be used to extend a line segment in any direction, and a compass can be used to draw a circle with any radius from the chosen center. The catch for this particular puzzle is that any points or lengths appearing in the final drawing must have been either present at the start or constructable from previously provided information.
To double a cubeโs volume, you start with its side length. Here that value might as well be 1 because it is the only unit of measurement given. To construct the larger cube, you have to figure out a way to draw one of its sides with the new required length, which is โ2 (the cube root of two), using just the straightedge and compass as tools.
It is a tough problem. For more than 2,000 years no one managed to solve it. Finally, in 1837, Pierre Laurent Wantzel explained why no one had succeeded by proving that it was impossible. His proof used cutting-edge mathematics of the time, the foundations of which were laid by his French contemporary รvariste Galois, who died at 20 in a duel that may have involved an unhappy love affair. At the ripe old age of 20 myself, I had achieved considerably less impressive mathematical accomplishments, but I at least understood Wantzelโs proof.
Here is the idea: Given a point as the origin and a length of distance 1, it is relatively straightforward to use the straightedge and compass to construct all points on a number line whose coordinates are rational numbers (ignoring, as mathematicians tend to do, the impossibility of actually plotting infinitely many points in only a finite amount of time).
Wantzel showed that if one uses only these tools, each newly constructed point must be a solution to a quadratic polynomial equation ax2 + bx + c = 0 whose coefficients a, b and c are among the previously constructed points. In contrast, the point โ2 is a solution to the cubic polynomial x3 โ 2 = 0, and Galoisโs theory of โfield extensionsโ proves decisively that you can never get the solution to an irreducible cubic polynomial by solving quadratic equations, essentially because no power of 2 evenly divides the number 3.
ChatGPT
The article from Scientific American by Emily Riehl, titled "Infinity Category Theory Offers a Birdโs-Eye View of Mathematics," explores how mathematicians have developed category theory into infinite dimensions, uncovering new relationships among mathematical concepts. Here's a brief summary:
Article Summary
Background in Mathematics:
Riehl begins with a personal anecdote from her college years, recalling a moment when she encountered a centuries-old mathematical puzzle while walking past a subway: the challenge of constructing a cube with double the volume of a given cube using only a straightedge and compass. This problem, known for over two millennia and associated with Plato, illustrates the limitations of classical geometrical tools.
Historical Context:
The puzzle's solution was proved impossible by Pierre Laurent Wantzel in 1837, using principles laid by รvariste Galois. Wantzel demonstrated that constructing the required cube root of two (โ2) using only a straightedge and compass is impossible because it involves solving a cubic polynomial, which cannot be reduced to solving quadratic equations through these simple tools.
Link to Infinity Category Theory:
The article transitions into discussing how these historical mathematical challenges relate to modern advances in category theory. Riehl suggests that just as Wantzel and Galois expanded the understanding of what can be achieved with basic geometric tools, modern mathematicians are expanding category theory into infinite dimensions. This expansion allows for new insights and connections across various mathematical areas, showing that seemingly simple tools and concepts can open pathways to solving more complex and abstract problems in modern mathematics.
Implications of Infinity Category Theory:
By moving category theory into infinite dimensions, mathematicians can view and solve problems from a broader and more unified perspective. This approach not only links diverse mathematical concepts but also simplifies the understanding of complex relationships in mathematics, akin to providing a "bird's-eye view" of the entire discipline.
The article highlights the evolution of mathematical thought from ancient geometry puzzles to contemporary theoretical frameworks, underscoring the continual growth and depth of the field. Emily Riehl uses the narrative to bridge historical mathematical developments with cutting-edge theoretical advances, illustrating the dynamic and interconnected nature of mathematical inquiry.
User
But an interesting question remains: How was I, a still-wet-behind-the-ears undergraduate in my third year of university study, able to learn to comfortably manipulate abstract number systems such as Galoisโs fields in just a few short weeks? This material came at the end of a course filled with symmetry groups, polynomial rings and related treasures that would have blown the minds of mathematical giants such as Isaac Newton, Gottfried Leibniz, Leonhard Euler and Carl Friedrich Gauss. How is it that mathematicians can quickly teach every new generation of undergraduates discoveries that astonished the previous generationโs experts?
None
Credit: Matteo Farinella
Part of the answer has to do with recent developments in mathematics that provide a โbirds-eye viewโ of the field through ever increasing levels of abstraction. Category theory is a branch of mathematics that explains how distinct mathematical objects can be considered โthe same.โ Its fundamental theorem tells us that any mathematical object, no matter how complex, is entirely determined by its relationships to similar objects. Through category theory, we teach young mathematicians the latest ideas by using general rules that apply broadly to categories across mathematics rather than drilling down to individual laws that apply only in a single area.
As mathematics continues to evolve, mathematiciansโ sense of when two things are โthe sameโ has expanded. In the past few decades many other researchers and I have been working on an extension of category theory to make sense of this new expanded notion of uniqueness. These new categories, called infinity categories (โ-categories), broaden category theory to infinite dimensions. The language of โ-categories gives mathematicians powerful tools to study problems in which relations between objects are too nuanced to be defined in traditional categories. The perspective of โzooming out to infinityโ offers a novel way to think about old concepts and a path toward the discovery of new ones.
CATEGORIES
Like many other mathematicians I know, I was drawn into the subject partly because of my poor memory. This confounds many people who remember high school mathematics as rife with formulas to memorizeโthe trigonometric identities come to mind. But I took comfort in the fact that the most commonly used formulas could be rederived from sin2ฮธ + cos2ฮธ = 1, which itself has an elegant geometric explanation: it is an application of the Pythagorean theorem to a right triangle with a hypotenuse of length 1 and an acute angle of ฮธ degrees.
This utopian vision of mathematics where everything just โmakes senseโ and nothing needs to be memorized falls apart to some extent at the university level. At that point students get to know the zoo of mathematical objects that have been conjured into existence in the past few centuries. โGroups,โ โringsโ and โfieldsโ belong to an area of mathematics known as algebra, a word derived from a ninth-century book by Persian mathematician and astronomer Muhammad ibn Musa al-Khwarizmi, the title of which is sometimes translated as The Science of Restoring and Balancing. Over the next millennium, algebra evolved from the study of the nature of solutions to polynomial equations to the study of abstract number systems. Because no real number x satisfies the equation x2 + 1 = 0, mathematicians built a new number systemโnow known as the complex numbersโby adding an imaginary number i and imposing the stipulation that i2 + 1= 0.
Algebra is only one of the subjects in a mathematics undergraduateโs curriculum. Other cornerstones include topologyโthe abstract study of spaceโand analysis, which begins with a rigorous treatment of the calculus of real functions before branching into the more exotic terrains of probability spaces and random variables and complex manifolds and holomorphic functions. How is a student supposed to make sense of it all?
ChatGPT
Emily Riehl's article continues to explore the remarkable ability of modern mathematicians to quickly assimilate and teach complex concepts that once baffled even the most brilliant minds in history. She ponders how, as an undergraduate, she was able to grasp abstract mathematical structures such as Galois fields in a relatively short time, a feat that would have astonished mathematical luminaries like Newton or Gauss. This rapid learning curve is attributed partly to the advancements in mathematics that provide a more unified and abstract perspective of the field, particularly through Category Theory.
Key Points from the Article:
Evolution of Mathematical Teaching and Understanding:
Mathematics education has transformed, enabling students to learn complex and abstract concepts more quickly than before. This is largely due to the development of frameworks like Category Theory, which offers a high-level view that simplifies the understanding of intricate relationships in mathematics.
Category Theory as a Pedagogical Tool:
Category Theory is crucial because it teaches that mathematical objects are defined by their relationships with similar objects. This approach allows educators to impart broad general rules that apply across various mathematical categories, rather than focusing narrowly on specific, isolated laws.
Infinity Categories (โ-categories):
The article discusses the extension of Category Theory into infinity categories, which manage to capture even more nuanced relationships between objects in what can be considered infinite dimensions. This new framework provides tools for addressing complex mathematical problems by viewing them from a broader perspective, essentially "zooming out to infinity."
Mathematical Foundations and Memory:
Riehl touches on the personal aspect of her journey into mathematics, noting that unlike the rote memorization often associated with high school math, higher mathematics relies more on understanding underlying principles. She illustrates this with the trigonometric identity derived from the Pythagorean theorem, which she finds more intuitive and less reliant on memory.
Challenges at the University Level:
Despite the foundational understanding provided by theories like Category Theory, the university-level study of mathematics introduces a "zoo" of mathematical objectsโgroups, rings, fields, etc.โthat belong to complex areas like algebra, topology, and analysis. These subjects demand a deep and nuanced understanding, which can be overwhelming for students.
Riehl's reflection on her educational experience highlights the evolution of mathematical thought and education, suggesting that modern advancements like Category Theory not only facilitate a deeper understanding of mathematics but also empower new generations to tackle advanced concepts more swiftly and effectively. This bridging of historical and contemporary mathematical challenges underscores the dynamic and continuously evolving nature of the field.
User
Open cybernetics systems I: feedback systems as optics
mattecapu category theory, publications May 26, 2021 13 Minutes
I coauthored my first two papers! [0] Itโs work Iโm very excited about, and it actually tells a single story which Iโd like to put together here, by sketching the bigger picture. Moreover, this is the same story behind my most recent talk, โGames with playersโ, so thereโs lots to talk about!
The papers stand in a theory-applications relationship. โTowards foundations of categorical cyberneticsโ lays the theoretical foundations for the second one, โTranslating from extensive form games to open games with agencyโ, though both works stand on their own. I wrote them with a bunch of MSP people, you can look them up on the arXiv.
I started blogging about the papers two weeks ago, and the size of what was supposed to me one post spiralled out of hand. Thereโs a lot ideas going around, and eventually I resorted to split the post in three parts plus a spin-off (which is actually an old resurrected draft):
Open cybernetic systems I: feedback systems as optics
Open cybernetic systems II: parametrised optics and agency
Open cybernetic systems III: control mechanisms and equilibria (coming soon)
Bonus: Amazing backward induction (coming soon)
Iโm going to introduce all the notions required to understanding our key idea, namely that parametrised optics are a good framework for formalising agency in open cybernetic systems. Itโs been known for a while that โvanillaโ optics do this for open dynamical systems (or more), so our contribution is really in the โparametrised = agencyโ part. Truth be told, thatโs also not completely new: notable precedents are โBackprop as a Functorโ, in which the Para construction is first sketched and Bruno Gavranoviฤโs recent work. Also, Toby Smithe has been playing around with a similar construction for active inference, which is intimately related to the theory of cybernetic systems.
Advertisements
REPORT THIS AD
In these articles Iโll assume familiarity with string diagrams (which, I argue, everybody is born with) and some basic categorical concepts, mainly from the world of monoidal categories. Sometimes Iโll stray away but those are points for the โadvanced readerโ, which can be safely skipped by the uninitiated.
Feedback systems as optics
In categorical systems theory, various kinds of โbidirectional morphisms with stateโ (which, we are about to see, is what optics means) have been used to represent dynamical systems. I argue a better word for whatโve been studying is feedback systems, since the kind of dynamics encoded by optics is that of action-reaction: in addition to the unfolding of an action, an optic also models a subsequent โreaction propagationโ step, where some kind of reward/loss/nudge (in a word: feedback) is returned to the system.
Contrast this with traditional dynamical systems, whose dynamics is encoded by operators acting on a space, a mathematical model representing action but devoid of feedback. Nevertheless, considerable attention is devoled to the study of observables of a system, i.e. a (usually scalar) quantity of interest which we monitor during the dynamics. In particular, one is often interested in how these quantities evolve as the system itself evolves, and thus a dynamical system equipped with a distinguished observable turns out to be very similar to a feedback system.
Another common occurrence is that the evolution of the systems itself is guided by one or more observables. Think of Hamiltonian mechanics, in which a functional h : X \to \mathbb R, defined on the space X of phases of a physical system, orchestrates the whole dynamics (together with a symplectic form on X). In these cases โfeedbackโ is an even more apt terminology.
The kind of feedback systems me and my group are most interested in are games and machine learning models. In both fields, action and feedback are equally important parts of the dynamics. In games, the โactionโ part is called play and the โfeedbackโ part is payoff distribution, often in the form of backward induction. In machine learning models, they are called the โforwardโ and โbackwardโ pass. The algorithm implementing the backward pass is backpropagation. Iโve written about the similarity between backward induction and backpropagation in the last post of this series (coming soon).
Advertisements
REPORT THIS AD
Nevertheless, Iโve already blogged about how backpropagation is secretely powered by the algebra of lenses. These are gadgets which pack bidirectional morphisms: a lens (X,S) \rightleftarrows (Y,R) is a pair of maps \mathrm{view}: X \to Y (or just โforward partโ) and \mathrm{update} : X \times R \to S (โbackward partโ), which live in some category with products \mathbf C. The terminology surrounding them comes from the functional programming community, where lenses are a rudimental abstraction for accessing and โmutatingโ data structures. [1]
Fig. 1: A lens
One can see the forward part as bringing about an action and the backward part as propagating a feedback. This is very evident in backpropagation, where the forward part of a lens represents a function being computed and the backward part is a reverse derivative being pulled back in order to propagate the loss gradient. Hence, for us, โdoโ and โpropagateโ (sometimes abbreviated to prop) are better terms for โviewโ and โupdateโ.
Whatโs quite important in the definition of lenses is that โpropagateโ has a dependency on X, the โstateโ. This fact (witnessed by the wire branching before {do} and going down to \mathrm{prop}) is actually enforced by the composition law of lenses:
In practical terms, this means that the feedback a lens propagates pertains to the computation actually happened, or that a lens, like the North, remembers.
Advertisement
This is made even more explicit in optics, a wide generalization of lenses. The leap in generality amounts to making the memory mechanism more expressive. Lenses remember exactly what they received from the environment in the form of a state, which is copied and preserved for the backward pass. In an optic, state is remembered, transmitted, and read out using a middleman, the residual. It is usually denoted by M, and features predominantly in the work we are doing, albeit on the sly [2]. This generalization also allows one to drop the assumption that \mathbf C is cartesian, and work with an arbitrary category instead. Still, we usually want to assume \mathbf C is at least monoidal, because it should stand for a category of systems, and monoidal categories allow the two most basic kinds of systems composition, sequential and parallel.
The memorization-transmission-readout mechanism is implemented through some clever mathematical machinery. First of all, residuals are assumed to live in their own category, the aptly-named and denoted category of residuals \mathbf M. It is itself monoidal, and acts on the category our optics are made of (\mathbf C), meaning that we can multiply a given M : \mathbf M with a given A : \mathbf C (pretty much like a scalar multiplication allows you to multiply numbers and vectors, i.e. objects of different sorts) [3]. We denote such a product M \bullet A.
A residual is attached to the codomain of the forward part and the domain of the backward part. An optic (X,S) \rightleftarrows (Y,R) then looks like a pair of maps \mathrm{do} : X \to M \bullet Y, \mathrm{propagate} : M \bullet R \to S. So the โdoโ part computes, from a given state in X, something in Y to give back to the environment and something in M to keep private. Then, given something in M (ideally, the readout of what we memorized in the forward pass) and some feedback in R coming from the environment, we can meaningfully propagate it to the environment. [4]
Notice that vertical wires now live in a different category than horizontal ones. I draw them blue for this reason. Ideally, these wires are not even drawn on the same plane, they live in the transverse dimension, going in and out of the place (this also the reason why the residual wire takes that long detour). This dimension will be greatly exploited in the next post, when Iโll introduce parametrised optics.
Advertisement
All in all, given a monoidal categories of residuals \mathbf M acting on a monoidal category \mathbf C, we get a monoidal category \mathbf{Optic}_\bullet(\mathbf C) whose objects are pair of objects of \mathbf C and morphisms are optics between them. Indeed, optics can be composed in sequence and in parallel:
The unit for \mathbf{Optic}(\mathbf C) is given by the pair (I,I), an aptly invisible object in the diagrammatic language. This language can be thought of as โliving insideโ \mathbf C diagrammatic language, though this is not completely true as we see from the fact there are wires coming from another category. String diagrams for optics are diagrams for so-called teleological categories.
ChatGPT
The blog post by Matteo Capucci, titled "Open cybernetic systems I: feedback systems as optics," explores the integration of categorical theory into the understanding of cybernetic systems, particularly through the concept of optics. Capucci co-authored two papers that delve into this theory and its applications to games and machine learning, proposing a novel way to formalize agency within open cybernetic systems using parametrized optics. Hereโs a concise summary:
Key Concepts and Ideas:
Feedback Systems as Optics:
Traditional vs. Optic Systems: Traditional dynamical systems focus solely on action without incorporating feedback. In contrast, optic systems in categorical theory include both action and feedback, representing systems as bidirectional morphisms with states.
Action-Reaction Dynamics: The dynamics encoded by optics include an action followed by a reaction (feedback), which is crucial for understanding systems that adapt or respond based on feedback, such as in games or learning models.
Applications to Games and Machine Learning:
In gaming, the action represents gameplay, and the feedback is the payoff or results that influence future decisions (e.g., through backward induction).
In machine learning, the forward pass (action) and the backward pass (feedback via backpropagation) are critical for adjusting models based on error gradients.
Parametrized Optics and Agency:
Capucci introduces the notion of parametrized optics as a framework for formalizing agency in cybernetic systems, an extension of the existing concept of optics used for open dynamical systems.
This framework is said to enhance the ability to model and analyze systems where agency and decision-making are integral, such as autonomous agents in AI or strategic players in economic models.
Algebra of Lenses:
The algebra of lenses, tools from functional programming, is employed to explain the bidirectional nature of optics. Lenses consist of a forward part ("do") and a backward part ("propagate"), facilitating both the execution of actions and the feedback mechanism.
Lenses remember the state (input), which influences how feedback is processed, echoing the memory function in many cybernetic systems.
Future Directions:
Capucci hints at exploring more complex structures called "parametrized optics" in subsequent posts, which will likely delve deeper into how these concepts can be applied more broadly in categorical cybernetics.
Diagrammatic Representation:
The use of string diagrams is emphasized for explaining these concepts, which are considered intuitive and natural for representing the flow and interactions within systems.
Capucci's work suggests a significant shift towards a more integrated and systematic approach in cybernetics, emphasizing the role of feedback and agency in modern systems theory. His approach using category theory not only abstracts the essence of interaction within systems but also provides a robust mathematical framework to analyze and predict the behavior of complex systems. This kind of theoretical groundwork is crucial for advancing fields like artificial intelligence, economics, and system biology, where understanding the underlying dynamics and feedback mechanisms can lead to more effective strategies and solutions.
User
Context
Insofar, Iโve spoken informally of โenvironmentโ, though itโs mathematical nature is of utmost importance. For a system, โenvironmentโ is everything that happens outside of its boundaries. More suggestively, everything is environment and to specify a system we cut a piece out. This mereological point of view will be greatly expounded in the next post, where weโll see that also agents arise by cutting a boundary.
For now, we limit ourselves to use this intuition to understand whatโs a context for an optic. A context is something you sorround an open system with to yield a closed system, i.e. something contained and finished in itself, whose dynamic can work without yielding to external parts.
Advertisement
This means that a closed system is necessarily of type (I,I) \rightleftarrows (I,I), a fact that manifests diagrammatically as an absence of wires going in or out:
Grey wires are invisible
Thus a context has to provide at least (a) an initial state and (b) a continuation, that is, something turning actions into feedbacks. These are respectively morphisms (I,I) \rightleftarrows (X, S) and (Y,R) \rightleftarrows (I,I), also known as states and costates:
The graphical depiction of costates makes it very obvious why they can be considered โcontinuationsโ: they turn around the information flow, switching from โactionโ mode to โfeedbackโ mode. While a costate amounts to two morphisms Y \to M \bullet I and M \bullet I \to R, you see how it can be easily converted into a single morphism Y \to R by composition. If \mathbf M = \mathbf C and \bullet = \otimes, then the two things are equivalent (that is, any function Y \to R can be made into a costate), but in general they are not: costates are only those morphisms that can be obtained by squeezing Y through a given residual, since this is the way the two parts of an optic can store and communicate information.
Slogan: โtime flows clockwiseโ.
Flow happening in \mathbf C in yellow, flow happening in \mathbf M in orange.
Itโd seem a state and a costate pair are enough to account for the environment, but thereโs still a subtlety. At the moment, the environment โceases to existsโ as soon as the system dynamics kicks in. That is, thereโs no way for the environment to store state independently from the system, while when a system is doing its thing, usually the rest of world still exists [5]. Hence we are missing something like this:
Labels are a bit confusing on this one: the red comb is what the totality of โsystemโs environment amounts to. The grey โenvironmentโ box is the residual dynamics of the environment, namely what happens in the environment while โsystemโ is doing its thing.
When we put everything together we realize that the data of a context is given exactly by a comb in the category of optics on \mathbf C. A comb is a U-shaped diagram, with a hole in the middle. The red piece above is a comb, whose hole is filled by the system.
Advertisement
This is funny because optics themselves are combs in \mathbf C, as illustrated by this beautiful animation made by Bruno:
Hence a compact way to define contexts for a system (X,S) \rightleftarrows (Y,R) is as โstates in optics of opticsโ (!!), i.e. combs in \mathbf{Optic}(\mathbf C) whose external boundaries are trivial (the unit) and whose internal boundaries are the ones of the system. [6]
This fits beautiful into the mereological picture of system and environment: a system is a hole in an environment, which โwrapsโ the system itself. Putting them together yields an inscrutable closed system. Also, let me stress again how boundaries of a system are a modelling choice. This is quite clear when we consider the composite of two systems: to each of the composee, the other one is part of the environment.
Variants & technological horizons
I canโt abstain from mentioning that, at the moment, two separate generalizations of โlensesโ are present in the literature. One is what I described above, known in the most general form as mixed optics or profunctor optics (these are an equivalent presentation of the same objects). The other one is F-lenses, which are themselves a generalization of dependent lenses aka containers aka polynomial functors.
This latter framework is quite important, especially as used in the work of Myers, Spivak, Libkind and others. Its strength lies in the fact they feature dependent types, which are very expressive and arguably the right way of doing certain things (i.e. mode-dependent dynamics). It also generalizes further in the direction of indexed containers, which in turn form the mathematical matter of Hancockโs interaction structures, perhaps the most conceptually sharp treatment of feedback systems around.
Advertisement
Dependently-typed mixed optics are thus the holy grail in this area and something me, Bruno, Jules (who blogged about it last year) and Eigil have been actively worked on in the last few months. They would allow the flexibility of optics, especially their indifference towards cartesian structure (very uncommon in resource theories) and at the same type the expressive power of dependent types. I hope weโll soon have good news on this front!
Finally, thereโs a pretty important bit that I swept under the rug in this article, which is that usually residuals are not kept explicit in optics. Optics are in fact defined as a quotient, using a coend indexed by residuals. The equivalence relation is generated by โslidingsโ:
From โOpen diagrams via coendsโ, by Mario Romรกn.
My impression is that something more should be said about this point. For example, thereโs merit in keeping the โhidden dynamicsโ of a context explicit. On the other hand, equivalence under sliding is a very reasonable condition. A way to resolve this tension is to turn the quotient into a groupoid, i.e. remember slidings as invertible 2-cells between optics. This fits very well with the philosophy behind the construction Iโll describe in the next post, Para.
Conclusions
I hope I managed to conveyed whatโs my intuition of feedback systems, namely as bidirectional morphisms whose mathematical incarnation is some flavour of optics. Residuals memorize information from the actions executed in the forward pass in order to effectively elaborate feedback in the backward pass. When a system is paired up with a context, it yields a closed system.
Next time, we are going to see how parametrised optics model agency in feedback systems. This will be a first step toward modelling cybernetic systems themselves, which are feedback systems with agency in a control loop.
Further reading
A list of further resources on this topic. Itโs probably gonna grow as things to add come to my mind.
Towards categorical foundations of learning, blog post by Bruno Gavranoviฤ, featuring more nice animations of optics and some insights I didnโt cover here.
A general definition of open dynamical system, talk by David Jaz Myers, featuring a lot of very cool mathematical insights.
Bayesian open games, paper by Jules Hedges and Jon Bolt, featuring full-blown optics-as-feedback-systems in the wild.
ChatGPT
In the blog post titled "Open cybernetic systems I: feedback systems as optics," Matteo Capucci delves into the integration of categorical theory with cybernetic systems, focusing on the conceptualization of 'environment' and its mathematical significance to systems theory. Here's a concise summary of the key points discussed:
Key Concepts and Ideas:
Defining the 'Environment':
Capucci emphasizes the mereological (part-whole relationship) perspective of systems where everything outside a system's boundary is considered the 'environment.' To define a system, one must effectively "cut out" a piece from the whole, framing the rest as its environment.
Contexts in Open Systems:
A context in systems theory surrounds an open system to transform it into a closed system, one that is self-contained and operates independently of external elements. This transformation is achieved by pairing initial states (states) and continuations (costates) that convert actions into feedbacks.
Costates and States:
Graphically depicted, costates function as continuations that reverse the flow of information, transitioning from action mode to feedback mode. They involve two morphisms that can be compressed into a single morphism, indicating a feedback loop from output back to input, crucial for adaptive systems.
The Role of Combs in Optics:
Capucci introduces the concept of a comb in the category of optics, a U-shaped diagram that encapsulates a system by defining both its external and internal boundaries. This structure helps in understanding how systems interact with their environments and manage information flow and feedback.
Future Directions and Technological Horizons:
Capucci points to ongoing developments in the generalization of lenses, such as mixed optics or profunctor optics, and F-lenses. These advancements are significant for incorporating dependent types, which offer higher expressiveness and handle mode-dependent dynamics effectively.
The discussion also touches on the mathematical formalism of residuals and coends in defining optics, suggesting that more nuanced treatments might keep the dynamics of a context more explicit.
Closing Systems:
A major theme is how a system, when combined with a context, forms a closed system. This closure is not just a theoretical construct but a practical framework for modeling systems that need to maintain a degree of independence from their environment.
Capucciโs post serves as an introductory exploration into how category theory can enrich our understanding of cybernetic systems through a sophisticated mathematical lens. The use of optics in this context underscores a deeper interplay between action and feedback in system dynamics, crucial for applications in control theory, artificial intelligence, and beyond. His upcoming posts promise to delve deeper into these concepts, particularly focusing on how parametrized optics model agency in feedback systems, marking a significant step toward modeling cybernetic systems themselves.
User
Compositional Game Theory
Neil Ghani Jules Hedges Viktor Winschel Philipp Zahn
AbstractโWe introduce open games as a compositional foundation
of economic game theory. A compositional approach potentially allows
methods of game theory and theoretical computer science to be applied
to large-scale economic models for which standard economic tools are not
practical. An open game represents a game played relative to an arbitrary
environment and to this end we introduce the concept of coutility, which
is the utility generated by an open game and returned to its environment.
Open games are the morphisms of a symmetric monoidal category and
can therefore be composed by categorical composition into sequential
move games and by monoidal products into simultaneous move games.
Open games can be represented by string diagrams which provide an
intuitive but formal visualisation of the information flows. We show that a
variety of games can be faithfully represented as open games in the sense
of having the same Nash equilibria and off-equilibrium best responses.
I. INTRODUCTION
The concept of compositionality is well-known and almost commonplace in computer science, where it is what ultimately allows
programmers to scale software to large systems. However, in many
other fields compositionality is essentially unknown and hence its
benefits are not available. In this paper we introduce compositionality
into a field where one might not believe it to be possible: the
study of strategic games and Nash equilibria. They are of interest in
economics and computer science where optimal decisions are taken
by interacting agents with conflicting goals.1
In contrast to classical game theory, where games are studied
monolithically as one global object, compositional game theory works
bottom-up by building large and complex games from smaller components. Such an approach is inherently difficult since the interaction
between games has to be considered. Moreover, in the compositional
approach, the equilibria of larger games should be defined from the
equilibria of the component games - but a priori, there is no reason
why this should be possible.
For example, in the prisonerโs dilemma game, each playerโs best
option is to defect, although, if they acted as a single agent, they
would cooperate. Moreover, if the one-shot prisonerโs dilemma game
is repeated, then cooperative equilibria become achievable. More
generally, the equilibria of a composite game are not necessarily
made up from those of the component games, and locally optimal
moves are not guaranteed to be globally optimal. In essence, game
theory contains emergent effects whereby a composite system exhibits
behaviours that are not (simple) functions of the behaviours of the
components. Accordingly, emergent effects make compositionality
very hard to achieve and the existence of a compositional model
of game theory is somewhat surprising. In order to arrive at this
goal we had to radically reformulate classical game theory from first
principles and rebuild it on open games.
Open games represent the relationship between different interactions in two dimensions: in sequence, if an interaction follows another
interaction and in parallel, if interactions take place simultaneously.
As such, we follow a path taken in the field of open systems [21], and
in particular categorical open systems [6] where compositional approaches to general systems are studied. Here, systems are modelled
1Games in the sense of game semantics are compositional, but they avoid
several difficult defining features of game theory by restricting to the 2-player
zero-sum setting.
as morphisms f : X โ Y in a symmetric monoidal category, where
the objects X and Y describe the boundaries of the open system,
where it interacts with its environment. This means that systems
f : X โ Y and f
โฒ
: X
โฒ โ Y
โฒ
can be composed in parallel using
the monoidal product to yield f โ f
โฒ
: X โ Xโฒ โ Y โ Y
โฒ
, and two
systems f : X โ Y and g : Y โ Z sharing a common boundary
can be composed sequentially by glueing along this boundary to yield
gโฆf : X โ Z. Ordinary, closed systems are recovered as scalars [1],
i.e. endomorphisms f : I โ I of the monoidal unit, which represents
a trivial boundary. Open games are accordingly the morphisms of a
symmetric monoidal category.
A compositional model of game theory does not only have to model
a game but also the interactions of the game with all other games
and environments. This can be seen as a form of continuation passing
style. This would still be hardly tractable if the environment of an
open game included arbitrary other open games. The crucial technical
feature underlying our approach is to describe the behaviour of an
open game relative to a simplified notion of an environment which we
call a context, in which the future is abstracted into a single utility
function. In this way, we reduce an arbitrarily complex game to a
set of individual decisions. The circularity of a Nash equilibrium,
where all players play mutually best replies, is finally handled by the
composition operators.
The theory of open games is based on two main predecessors.
Firstly, in [17] games are defined as processes and in [2] the dynamics
but not the equilibria are treated compositionally. The second predecessor is the theory of higher order games and selection functions,
for example in [4] and [12], which give a theory of equilibria relative
to an environment but are not strongly compositional. Selection
functions can be used to model goals of agents compositionally [11].
Combining features of these approaches into a single mathematical
object required the innovations mentioned above and led us to
discover the idea of an open game. After we developed open games,
connections to lenses and the geometry of interaction were noticed
respectively by Jeremy Gibbons and Tom Hirschowitz.
We omit proofs in this paper, which can be found in [8] and [10].
We also work over the category of sets to keep notation and overheads
to a minimum โ for a full categorical account, see once more [8]. The
rest of this paper is structured as follows: The next section introduces
selection functions as a key ingredient to open games. Section III
introduces the definition of an open game and discusses its elements,
followed by some examples in Section IV. The monoidal category
of open games is introduced in Section V and the string diagrams
attached to this category in Section VI. We then turn to examples built
compositionally: in Section VII we discuss simultaneous move games
and in Section VIII sequential move games. Section IX concludes the
paper with an outlook on further work.
ChatGPT
The paper "Compositional Game Theory" by Neil Ghani, Jules Hedges, Viktor Winschel, and Philipp Zahn, introduces the concept of open games as a foundational approach to game theory that is inherently compositional. This approach allows for the application of methods from game theory and theoretical computer science to large-scale economic models, which are often too complex for standard tools. Hereโs a summary of the key points discussed in the paper:
Key Concepts and Ideas:
Introduction of Open Games:
Open games are conceptualized as games that interact with an arbitrary environment. They represent a game's utility (referred to as "coutility") that is generated by the game and returned to its environment.
Compositionality in Game Theory:
Unlike traditional game theory that views games as monolithic entities, compositional game theory builds complex games from smaller, simpler components. This approach aligns with practices in computer science where systems are built by combining smaller modules.
Symmetric Monoidal Category:
Open games are described as morphisms within a symmetric monoidal category, which allows them to be composed either sequentially for turn-based games or in parallel for simultaneous move games.
Visual Representation with String Diagrams:
The use of string diagrams provides a formal yet intuitive way to visualize the flow of information within these games, helping to clarify how games interact within their environment.
Challenges of Compositionality:
The paper discusses the inherent difficulties of ensuring that the equilibria of a composite game reflect those of its components due to emergent behaviors that are not simply the sum of individual parts.
Utility Function and Nash Equilibria:
Open games simplify the complex interactions of a game into a set of individual decisions abstracted into a single utility function, facilitating the analysis of Nash equilibria where each player's strategy is a best response to the others.
Theoretical Foundation and Innovations:
The development of open games required a radical rethinking of classical game theory, integrating concepts like selection functions, which model the goals of agents, and connecting these to broader mathematical structures like lenses and geometry of interaction.
Applications and Examples:
The paper discusses how open games can be applied to various types of economic and strategic scenarios, including simultaneous and sequential move games, illustrating the practical utility of this theoretical framework.
Conclusion:
The concept of open games introduces a significant shift in how game theory can be applied, particularly in economic modeling and strategic decision-making. By leveraging the principles of compositionality, this approach offers a scalable framework that can handle the complexity of modern economic systems more effectively than traditional methods. The paper promises further exploration of this innovative approach, suggesting a rich field of study that bridges theoretical computer science and economics.
User
The Compositional Structure of Bayesian Inference
Dylan Braithwaite
Department of Computer and Information Sciences, University of Strathclyde, Glasgow, UK
Jules Hedges
Department of Computer and Information Sciences, University of Strathclyde, Glasgow, UK
Toby St Clere Smithe
Topos Institute, Berkeley, CA, USA
Department of Experimental Psychology, University of Oxford, UK
Abstract
Bayesโ rule tells us how to invert a causal process in order to update our beliefs in light of new
evidence. If the process is believed to have a complex compositional structure, we may observe that
the inversion of the whole can be computed piecewise in terms of the component processes. We
study the structure of this compositional rule, noting that it relates to the lens pattern in functional
programming. Working in a suitably general axiomatic presentation of a category of Markov kernels,
we see how we can think of Bayesian inversion as a particular instance of a state-dependent morphism
in a fibred category. We discuss the compositional nature of this, formulated as a functor on the
underlying category and explore how this can used for a more type-driven approach to statistical
inference.
2012 ACM Subject Classification Theory of computation โ Categorical semantics; Mathematics of
computing โ Probabilistic representations; Mathematics of computing โ Bayesian computation
Keywords and phrases monoidal categories, probabilistic programming, Bayesian inference
Digital Object Identifier 10.4230/LIPIcs.MFCS.2023.24
Related Version This paper contains material from unpublished preprints [16] and [3].
Extended Version: https://arxiv.org/abs/2305.06112
Funding Dylan Braithwaite: This author is funded through an NPL Industrial CASE Studentship.
1 Introduction
A Markov kernel f is a function whose output is stochastic given the input. It can be thought
of as a stateless device which, if supplied with inputs, will produce an output that depends
probabilistically on the input. Mathematically speaking, this is nothing but a conditional
distribution P(y|x). The problem of Bayesian inversion is that we observe the output of
such a device but only have a probabilistic โpriorโ belief about the input, and we would like
to update our beliefs about what the input was given the output. For example, suppose
a process takes the roll of an unseen die and tells us whether it was even or odd, except
with probability 10% it flips the result. If the process tells us that a specific roll resulted in
โevenโ then Bayesโ law tells us how we should update a prior belief that the die is uniformly
random, to obtain a belief about what the specific roll was.
It is commonly the case that the process is not a black box, but is a composite process
formed from simpler pieces. The conditional distribution formed by sequentially composing
two such stochastic functions is given by forming their joint distribution and then marginalising
over the middle variable. In the case of a long chain of sequentially composed functions,
belief updating for the whole can be done โcompositionallyโ in terms of belief updating for
each individual part, in a process notably similar to backpropagation in neural networks. A
process such as this underlies probabilistic programming languages, which are able to run
programs โbackwardsโ after conditioning on some output.
ChatGPT
The paper titled "The Compositional Structure of Bayesian Inference" by Dylan Braithwaite, Jules Hedges, and Toby St Clere Smithe explores the application of compositional methods to Bayesian inference, particularly through the lens of category theory and functional programming. The authors provide a nuanced examination of how Bayesian processes, typically structured as sequences of probabilistic events or Markov kernels, can be decomposed and analyzed using compositional techniques. Hereโs a summary of the key points discussed in the paper:
Key Concepts and Ideas:
Bayesian Inversion and Markov Kernels:
The core concept revolves around Bayesian inversion, where the output of a stochastic function (or Markov kernel) is observed, and the goal is to infer the probabilistic input based on this output. Markov kernels are modeled as stateless devices that produce outputs dependent on their inputs according to a conditional probability distribution
๐
(
๐ฆ
โฃ
๐ฅ
)
P(yโฃx).
Compositional Structure in Bayesian Processes:
Unlike traditional views of Bayesian processes as monolithic black boxes, the paper posits that these processes often have a compositional structure. This means that a complex process can be broken down into simpler, component processes whose interactions define the overall system behavior.
Sequential Composition and Bayesian Updating:
The authors discuss how the composition of Bayesian processes can be modeled by sequentially composing stochastic functions, akin to forming a chain of conditional distributions that are then simplified through marginalization. This approach allows for updating beliefs about the system in a piece-by-piece fashion, similar to how backpropagation works in neural networks.
Categorical and Functorial Framework:
The paper introduces a categorical framework to model these compositions, using a category of Markov kernels. Bayesian inversion within this framework is treated as a state-dependent morphism in a fibred category, with the entire compositional structure formulated as a functor over the underlying category. This formalism supports a more rigorous and structured approach to dealing with complex Bayesian systems.
Implications for Probabilistic Programming:
The insights from this compositional approach have practical implications for probabilistic programming languages, which are designed to execute programs in a manner that conditions on outputs. The ability to "run programs backward" by updating beliefs after observing outputs is enhanced by understanding the compositional structure of the underlying Bayesian processes.
Contribution and Novelty:
The paper is notable for integrating concepts from category theory and functional programming with Bayesian inference, offering a new perspective on how these processes can be decomposed and analyzed. This approach provides a systematic method for understanding and implementing Bayesian inference in complex systems where direct application of traditional methods might be impractical or inefficient.
By framing Bayesian inference within a categorical and compositional context, the authors pave the way for more sophisticated methods in computational statistics and probabilistic programming, potentially leading to more powerful and flexible probabilistic modeling tools.
This paper represents a significant step toward bridging the gap between theoretical computer science and practical statistical inference, offering new tools and perspectives for researchers and practitioners in both fields.
User
Example: Estimating Transition Probabilities in Markov Chains
To illustrate the compositional nature of Bayesian inversion in practice we consider how to
view a typical Bayesian inference problem in this framework. Namely we use as our example,
the common problem of learning transition probabilities for a Markov chain from an observed
sequence of states [21].
In the category FinStoch, a Markov chain is nothing but an endomorphism t : S โ S
for some finite set S of states, with an initial state distribution s : I โ S. Often we do not
know the actual transition probabilities, but instead have a family of distributions dependent
on another external parameter, t : S โ ฮ โ S. In such a situation we want to choose an
optimal value of ฮ based on a sequence of observations of states in the past. A common
technique for choosing this value is to consider the observed state sequence as an element of
the space S โ . . . โ S, with which we can perform a Bayesian update. That is, given a state
sequence (s1, . . . , sn) we would like to compute a posterior distribution P(ฮ|s1, . . . , sn).
From the data involved we can define a family of maps fn : ฮ โ S
โn describing the
distribution over n-length state sequences corresponding to any value of ฮ (see Figure 2 for
an example). Computing the Bayesian inverse of fn we obtain kernels which map sequences
S
โn to posterior distributions over ฮ. Of course, as we have already discussed, in order to
compute this all we need is the Bayesian inverse for the transition matrix t. Knowing the
functorial relationship between โforwardโ models fn and their corresponding inverse models,
we can build an inverse for fn out of many copies of the inverse for t.
f4 =
t
t
t
s
ฮ
S
S
S
S
Figure 2 An example of a โstate-traceโ kernel obtained from a Markov chain, yielding state
sequences of length 4.
M FC S 2 0 2 3
24:12 The Compositional Structure of Bayesian Inference
Figure 2 depicts a typical composite which we may want to invert. It consists only of
four kinds of non-trivial cell: the transition matrix t, the initial state distribution s, copy
maps copyS : S โ S โ S and copyฮ : ฮ โ ฮ โ ฮ, and a delete map deleteฮ : ฮ โ I. As t
is an arbitrary kernel there is little we can say about its Bayesian inverse, but we can fully
characterise the inverses for the other kinds of cells. The inverse of s at p should have a
type s
โฏ
p
: Sp โ I, but by the uniqueness of the delete map this is equal to deleteSp
: Sp โ I.
Dually, filling in the definition of Bayesian inversion for deleteฮ at p requires that (deleteฮ)
โฏ
p
is exactly p : I โ ฮ.
Finally we consider the Bayesian inverse of copyS. Of course the case for copyฮ is identical.
In the case of non-dependent Bayesian lenses, without supports, it is difficult to understand
what these copy maps do. The copy operation is supported on only a small subset of its
domain. Intuitively this is the diagonal set {(s, s)|s โ S}, but in the abstract case this set
notation is not valid. We may wonder if there is a way that we can describe this using only
the axiomatic setting we have been working with so far. Indeed, we can show that there is in
fact an isomorphism of the support object (S โ S)copyฮโฆp
โผ= Sp and moreover, the morphism
witnessing this isomorphism is exactly the Bayesian inverse of copy.
This isomorphism is a very powerful fact. It enforces in the type of a morphism, using the
notion of dependent typing present in DBLens(C), that the observations we make from a
process involving copying should preserve the exact equalities expected given our knowledge
about the generative processes from which the observations originated.
It is not difficult to derive an explicit formula for this with traditional probability theory,
but our structural viewpoint can offer a more pedagogical approach to describing inference
algorithms. Using a 2-dimensional syntax for kernels and understanding the compositional
nature of Bayesian inversion, a lot of the apparent complexity of equations in statistics
is clarified by applying operations piecewise on string diagrams. It additionally becomes
straightforward to see how to extend this to more complicated scenarios. For example, a
hidden Markov model is obtained from the above example simply by postcomposing each
wire with an additional kernel o : S โ O. Then compositionality described exactly how to
extend a solution to account for the additional mapping.
This approach provides not just additional pedagogy, but also is very amenable to
algorithmic study: as future work, we hope to investigate ways in which, by isolating the
basic repeated units and their compositional structure, we can automatically generate a lot
of the additional data required and can perhaps facilitate certain optimisations.
7 Further work
Our work situates Bayesian inversion amongst a range of bidirectional processes observed
in different contexts, each of which exhibit lens (or lens-like) structure and โcyberneticโ
application [4]: reverse-mode automatic differentiation (generalizing backpropagation of
error) [6]; economic games [9]; reinforcement learning [11]; and database updating [7]. It
was in this latter setting, in the context of functional programming, that lenses were first
described, and generalizations of lenses remain in popular use in that setting. This points to a
first avenue for future work: a new approach to probabilistic programming that incorporates
both Bayesian and differential updating, extending the currently popular use of deterministic
lenses, and with general application to cybernetic systems.
Probabilistic programming languages allow the programmer essentially to construct
Markov kernels in a compositional way using a standard programming language syntax,
and perform inference on the resulting models [20]. Typically, performing inference is not
D. Braithwaite, J. Hedges, and T. St Clere Smithe 24:13
transparently compositional, and so we hope that our results will lead to improvements
here, for instance by allowing the programmer to โamortizeโ parts of an inference process.
Perhaps more importantly, our approach could lead to the use of a dependent type system
to statically rule out events of measure zero. Furthermore, in a generically bidirectional
language of the kind we envisage, we expect that compilers will be able to share optimizations
across differential and probabilistic parts.
We expect these ideas not only to be of use to language designers therefore, but also to
users. In many applications of probabilistic inference, one first constructs a โjoint modelโ: a
joint state across all the variables of interest, which may factorise according to a Bayesian
network or other graphical model. Because computing exact Bayesian inversions involves
marginalization (normalization: the sum or integral in the Chapman-Kolmogorov equation),
and such a computation is often intractable, one usually resorts to approximate methods,
and this is where the โextraโ morphisms in categories of Bayesian lenses come in: they can
be understood as approxmate inverses.
By parameterizing these lenses, one can associate to them loss functions that characterize
โhow farโ a given inversion is from optimality2
; like the inversions themselves, these losses
depend on both the priors and the observations, and they again compose according to a lens
pattern. The third author, in the unpublished works [17, 18], has begun developing this
programme, based on ideas from compositional game theory [9]. It results in an account
of approximate inference whose algorithms are โcorrect by constructionโ, which may be
an advantage over traditional methods which simply start from a given joint distribution.
Moreover, because the loss functions are โlocalโ to each lens, the resulting framework captures
the compositional structure seemingly exhibited by โpredictive codingโ neural circuits in
the brain [1]. Similarly, by making use of the lens structure presented here, the framework
suggests a formal unification of backprop and predictive coding, as sought by various authors
[13, 15], and it reveals connections to the method of backward induction in reinforcement
learning [11]. We hope that future developments make use of these relationships, so that we
may build intelligent systems that are both efficient and well-understood.
ChatGPT
The section from the paper "The Compositional Structure of Bayesian Inference" delves into the practical application of compositional Bayesian inversion, specifically in learning transition probabilities for a Markov chain from an observed sequence of states. This example illustrates the power of compositional methods in handling complex probabilistic models. Hereโs a detailed breakdown of the key points discussed in this section:
Key Concepts and Ideas:
Markov Chains and Transition Probabilities:
A Markov chain is represented as an endomorphism
๐ก
:
๐
โ
๐
t:SโS in the category FinStoch, where
๐
S is a finite set of states and
๐
:
๐ผ
โ
๐
s:IโS is the initial state distribution. Typically, the transition probabilities are not known directly but are parameterized by an external parameter, leading to a transition matrix
๐ก
:
๐
โ
ฮ
โ
๐
t:SโฮโS.
Bayesian Inversion for Learning Parameters:
The task is to update beliefs about the parameter
ฮ
ฮ based on observed sequences of states. Given an observed sequence, Bayesian inference is used to compute a posterior distribution
๐
(
ฮ
โฃ
๐
1
,
.
.
.
,
๐
๐
)
P(ฮโฃs
1
โ
,...,s
n
โ
). The sequences are treated as elements of the space
๐
โ
.
.
.
โ
๐
Sโ...โS.
Compositional Bayesian Inversion:
The paper details how the Bayesian inverse of a function
๐
๐
:
ฮ
โ
๐
โ
๐
f
n
โ
:ฮโS
โn
, which describes the distribution over state sequences for any value of
ฮ
ฮ, can be computed. This inverse maps sequences
๐
โ
๐
S
โn
to posterior distributions over
ฮ
ฮ. The compositional nature means that the inverse for the entire sequence can be built out of many copies of the inverse for the transition matrix
๐ก
t.
Diagrammatic Representation:
An example is provided with a "state-trace" kernel diagram, showing how state sequences of length 4 are processed in this framework. The diagram includes components like the transition matrix, the initial state distribution, copy maps, and delete maps, illustrating the flow and transformation of information through the Markov chain.
Implications for Probabilistic Programming:
This approach aligns with the mechanisms underpinning probabilistic programming languages, which can execute programs 'backwards' by conditioning on outputs. This highlights the potential for applying these compositional techniques to enhance probabilistic programming, making the inference processes more efficient and interpretable.
Practical Implications:
This example underscores the utility of compositional Bayesian methods in simplifying and structuring the process of learning and inference in complex probabilistic models like Markov chains. By breaking down the process into manageable parts and using category theory to systematically approach the problem, the authors demonstrate a clear pathway from theoretical foundations to practical implementation.
The compositional approach not only aids in the conceptual understanding of Bayesian inference but also provides a robust framework for developing algorithms that can handle the complexities of real-world data efficiently. This is particularly beneficial in fields where the underlying models are inherently sequential and interconnected, such as in sequence analysis, speech recognition, or predictive text systems.
Overall, this section of the paper highlights how advanced mathematical frameworks, when applied thoughtfully, can significantly enhance our ability to model, understand, and predict based on complex data structures.
User
A list of everything we've talked about so far.
ChatGPT
Sure, hereโs a comprehensive list of topics we've discussed so far:
Category Theory:
The general concept of Category Theory as a unifying framework in mathematics.
Applications of Category Theory in various domains such as computer science, algebra, and topology.
Functional Programming and Abstract Algebra:
Importance of understanding functional programming and abstract algebra before delving into Category Theory.
Recommended books and resources for learning functional programming and abstract algebra.
Visual Representation in Category Theory:
The significance of visual tools like diagrams in understanding and exploring Category Theory concepts.
Examples of visual representations like string diagrams used to convey complex relationships in mathematical structures.
Open Cybernetic Systems:
The concept of open cybernetic systems as discussed in Matteo Capucci's blog post.
Feedback systems as optics and parametrized optics for formalizing agency in open systems.
Compositional Game Theory:
Introduction of open games as a compositional foundation of economic game theory.
The role of symmetric monoidal categories in the compositional structure of games.
Bayesian Inference and Compositional Structures:
The compositional structure of Bayesian inference as a means to simplify complex probabilistic models.
Markov chains and Bayesian inversion in the context of learning transition probabilities from observed data.
Educational and Practical Applications:
Use of Category Theory in practical scenarios like cybernetic systems and game theory.
Implications of compositional Bayesian methods for probabilistic programming and other applied domains.
These topics cover a range of mathematical and computational theories and applications, showcasing how advanced concepts in mathematics can be effectively used to address complex problems in various scientific and engineering fields.
User
Could I get some analogies and metaphors to help explain these?
ChatGPT
Certainly! Using analogies and metaphors can make complex topics like these more relatable and easier to understand. Here are some analogies for the topics we discussed:
Category Theory:
Library System Analogy: Think of Category Theory as the organizing system of a library where books (objects) are connected by references (morphisms). Just as a reference links one book to another, showing how they are related, morphisms link different mathematical structures, highlighting their relationships.
Functional Programming and Abstract Algebra:
Lego Building Blocks Analogy: Functional programming and abstract algebra can be compared to building structures with Lego blocks. Each function or algebraic rule is like a Lego block, and building complex programs or mathematical models is like constructing a Lego masterpiece, where each piece must fit perfectly with others based on specific rules.
Visual Representation in Category Theory: