new notes

3kb7-dispersion.md

@@ -1,5 +1,5 @@
 # Dispersion
-#centraltendency #math #statistic #estimative #theory
+#centraltendency #math #statistics #estimative #theory
 
 - It is the **second** dimension
 - How the data is dispersed, it is more clustered or spread out

7hky-python-variable-scope.md

@@ -0,0 +1,43 @@
+# Python Variable Scope
+#python #rules #programming #scope
+
+- Variable has scope, and exist precedence regarding the local of reading values (local then global)
+- If you want to read global variables with variable local with the same name is possible to use `global <variable
+  name>`. Example below:
+
+```python
+>>> b = 6
+>>> def f3(a):
+...     global b
+...     print(a)
+...     print(b)
+...     b = 9
+...
+>>> f3(3)
+3
+6
+>>> b
+9
+```
+
+- [[ig5o-closures]]# is a form to deal with **Free Variables**
+
+## Logic Lookup
+
+
+- There is a global `x` declaration, `x` comes from and is assigned to the `x` global
+variable module.
+- There is a non-local `x` declaration, `x` comes from and is assigned to the `x` local
+variable of the nearest surrounding function where x is defined.
+- `x` is a parameter or is assigned a value in the function body, then `x` is the local
+variable.
+- If `x` is referenced but is not assigned and is not a parameter:
+	- `x` will be looked up in the local scopes of the surrounding function bodies
+(non-local scopes).
+	- If not found in surrounding scopes, it will be read from the module global
+scope.
+	- If not found in the global scope, it will be read from `__builtins__.__dict__`
+
+### References
+- Ramalho, 2022, 308-311
+- Ramalho, 2022, 315-317

aswx-python-decorator-implementation-with-parameters.md

@@ -0,0 +1,33 @@
+# Python Decorator Implementation with Parameters
+#designpattern #python #decorator
+
+```python
+import time
+
+DEFAULT_FMT = '[{elapsed:0.8f}s] {name}({args}) -> {result}'
+
+def clock(fmt=DEFAULT_FMT):
+    def decorate(func):
+        def clocked(*_args):
+            t0 = time.perf_counter()
+            _result = func(*_args)
+            elapsed = time.perf_counter() - t0
+            name = func.__name__
+            args = ', '.join(repr(arg) for arg in _args)
+            result = repr(_result)
+            print(fmt.format(**locals()))
+            return _result
+        return clocked
+    return decorate
+
+if __name__ == '__main__':
+    @clock()
+    def snooze(seconds):
+        time.sleep(seconds
+```
+
+- `**locals` means that get all variables locals and transform it to the following line:
+	- `elapsed=elapsed, name=name, args=args, result=result`
+
+### References
+- Ramalho, 2022, p322-323

cea0-decorators.md

@@ -0,0 +1,70 @@
+# Python Decorators
+#python #designpattern #programming #decorator
+
+- Is a type of #[[dcdy-callable-objects]]
+- The objective: Decorate another function and take this function as argument
+- Most part of the decorators change the function decorated
+- Decorators are defined in one module and used and other modules as well
+- They run on the step of `import time`
+
+```python
+@decorate
+def target():
+	print("running target")
+```
+- Code above transform into: `target = decorate(target)`
+
+```python
+#file registration.py
+registry = []
+def register(func):
+    print(f'running register({func})')
+    registry.append(func)
+    return func
+@register
+def f1():
+    print('running f1()')
+@register
+def f2():
+    print('running f2()')
+def f3():
+    print('running f3()')
+def main():
+    print('running main()')
+    print('registry ->', registry)
+    f1()
+    f2()
+    f3()
+if __name__ == '__main__':
+    main()
+
+$python registration.py
+running register(<function f1 at 0x100631bf8>)
+running register(<function f2 at 0x100631c80>)
+running main()
+registry -> [<function f1 at 0x100631bf8>, <function f2 at 0x100631c80>]
+running f1()
+running f2()
+running f3()
+```
+
+```python
+>>> import registration
+running register(<function f1 at 0x10063b1e0>)
+running register(<function f2 at 0x10063b268>)
+>>> registration.registry
+[<function f1 at 0x10063b1e0>, <function f2 at 0x10063b268>]
+```
+- [[wmjn-python-decorator-implementation]]#
+- [[aswx-python-decorator-implementation-with-parameters]]#
+- [[naei-cache-lru-cache-singledispatch]]#
+- [[hl16-parameterized-decorators]]#
+
+> [!tip] Quote
+> * A decorator is a function or another callable<br>
+> * A decorator may replace decorated function with a different one<br>
+> * Decorators are executed immediately when a module is loaded<br>
+> Ramlho, 2022, p306
+
+### References
+- Ramalho, 2022, p302-307

cj29-random-sampling-and-random-bias.md

@@ -0,0 +1,16 @@
+# Random Sampling and Sample Bias
+#statistics #math #sampling #modelling
+
+- Data Scientist must focus on the data in hand and the sampling method
+- If a sample comes from physical process, would be good to know which process is to facilitate modelling
+	- We can make some assumptions here, because physical process obey some physical laws and rules
+	- For example: flip a coin obey a physical process, and it is binomial distribution (head or tails)
+		- We can model other problems as flip a coin or binomial phenomena (fraud or not, buy or not)
+- Try to create samples that can represent the population of the data (hard task)
+- Small datasets are better to plot and manipulate. Focus on the **good** data.
+- **Missing/Outliers** are important when we are dealing with **Big Data**
+- Massive data are important when they are sparse, like a matrix of terms on search query algorithm
+- Sample mean: x, Population mean: u
+- Stratified sample can be hard because we need to know which group is important to divided and take into account
+- [[vkim-selection-bias]]# A systematic way to commit mistakes when sampling from population making the measurement process not
+  confident for our purpose. It has a specific direction to point!

dcdy-callable-objects.md

@@ -10,6 +10,7 @@
 - They are all called with `()`
 - How to know if object is callable: `callable()`
 - How to type callable objects: [[80tb-typing-callable]]#
+- [[cea0-decorators]]# is a type of callable object
 
 ```python
 import random

h92w-sampling-distribution.md

@@ -0,0 +1,16 @@
+# Sampling Distribution
+#theory #math #statistics #distribution #sampling #normaldistribution
+
+- It is the distribution of sample statistic like mean for example
+	- Sample 5 numbers from data and calculate the mean is an example of sample statistic
+	- Sample 20 numbers from data and calculate the mean is another example
+	- Make it multiple times and, you have the **sampling distribution**
+- It is bell-shaped
+	- This phenomenon is called [[lh9b-central-limit-theorem]]
+- Variability error on the sampling distribution: SE=$\frac{S}{\sqrt{n}}$
+	- To compute SE can be expensive, for that, we use [[s6vc-bootstrapping]]#
+		- Bigger samples give less SE
+		- It is different from `standard deviation`
+
+### References
+- Bruce, 2017, p57-61

hl16-parameterized-decorators.md

@@ -0,0 +1,50 @@
+# Parameterized Decorators
+#python #decorator #designpattern #programming
+
+- To create a decorator with parameters is necessary to build a **decorator factory**
+	- This decorator factory will accept parameter and pass to the decorator itself
+
+```python
+registry = set()
+def register(active=True):
+    def decorate(func):
+        print('running register'
+              f'(active={active})->decorate({func})')
+        if active:
+            registry.add(func)
+        else:
+            registry.discard(func)
+        return func
+    return decorate
+@register(active=False)
+def f1():
+    print('running f1()')
+@register()
+def f2():
+    print('running f2()')
+def f3():
+    print('running f3()')
+
+>>> from registration_param import *
+running register(active=False)->decorate(<function f1 at 0x10073c1e0>)
+running register(active=True)->decorate(<function f2 at 0x10073c268>)
+>>> registry
+{<function f2 at 0x10073c268>}
+>>> register()(f3)
+running register(active=True)->decorate(<function f3 at 0x10073c158>)
+<function f3 at 0x10073c158>
+>>> registry
+{<function f3 at 0x10073c158>, <function f2 at 0x10073c268>}
+>>> register(active=False)(f2)
+running register(active=False)->decorate(<function f2 at 0x10073c268>)
+<function f2 at 0x10073c268>
+>>> registry
+{<function f3 at 0x10073c158>}
+```
+
+- Code above use the concept of [[ig5o-closures]] and has normal call as well `register()(f3)`
+- [[wmjn-python-decorator-implementation]] but with parameters:
+  [[aswx-python-decorator-implementation-with-parameters]]#
+
+### References
+- Ramalho, 2022, p329-335

ig5o-closures.md

@@ -0,0 +1,62 @@
+# Closures
+#python #programming #scope
+
+- Closures uses **Free Variables**
+	- They are non-local neither non-global
+	- They come from external functions
+- They are made using nested functions
+
+```python
+def make_averager():
+    series = [] # mutable free variable
+    def averager(new_value):
+        series.append(new_value)
+        total = sum(series)
+        return total / len(series)
+    return averager
+
+>>> avg = make_averager()
+>>> avg(10)
+10.0
+>>> avg(11)
+10.5
+>>> avg(15)
+12.0
+```
+
+- But when you dealing with immutable variables, a keyword `nonlocal` is necessary to identify the free variables
+  outside of local scope
+	- When the immutable variable changes, a new variable is created, but local. However, this local variable does not
+	  exist.
+
+```python
+def make_averager():
+    count = 0
+    total = 0
+    def averager(new_value):
+        nonlocal count, total
+        count += 1
+        total += new_value
+        return total / count
+    return averager
+
+>>> avg = make_averager()
+>>> avg(10)
+Traceback (most recent call last):
+  ...
+UnboundLocalError: local variable 'count' referenced before assignment
+>>>
+
+def make_averager():
+    count = 0
+    total = 0
+    def averager(new_value):
+        nonlocal count, total # Fix the problem above
+        count += 1
+        total += new_value
+        return total / count
+    return averager
+```
+
+### References
+- Ramalho, 2022, p311-314

lh9b-central-limit-theorem.md

@@ -0,0 +1,8 @@
+# Central Limit Theorem
+#statistics #math #sampling #distribution #normaldistribution
+
+- A [[h92w-sampling-distribution]] is bell-shaped, will be normally distributed, or in other words, it is Gaussian.
+	- It happens even when the distribution of the raw numbers is not Gaussian
+	- The data need to be big enough or go to the infinite
+- Given that, it is possible
+	- T-distribution to calculate the [[h92w-sampling-distribution]] for inference

mj9x-degrees-of-freedom.md

@@ -1,5 +1,5 @@
 # Degrees of Freedom
-#statistic #math #theory #bias
+#statistics #math #theory #bias
 
 - Variance formula: $s^2 = \frac{1}{n-1} \sum_{i=1}^{n} (x_i - \overline{x})^2$
 - The formula above has $\frac{1}{n-1}$ which means degrees of freedom.

naei-cache-lru-cache-singledispatch.md

@@ -0,0 +1,44 @@
+# cache, lru-cache, singledispatch
+#python #efficiency #decorator #designpattern #programming
+
+## Cache
+
+- Save call results in a `dict` structure, therefore make the process more fast reusing last results
+- It for fast commands, because it can fill all the memory because it is unlimited. Another option is `lru_cache` with
+  `max_size`
+- They are good options for remote API
+- All the arguments of the functions need to be hashables ([[nxj2-hashable]]) because they will be save in a `dict`
+```python
+import functools
+
+from clockdeco import clock
+
+@functools.cache # stacked decorators works like that cache(clock(fibonnaci))
+@clock
+def fibonacci(n):
+	if n < 2:
+			return n
+		return fibonacci(n - 2) + fibonacci(n - 1)
+
+if __name__ == '__main__':
+    print(fibonacci(6))
+```
+
+## lru-cache
+
+- Similar to `cache` but has `typed` and `max_size`
+- *least-recently-used*: The old ones that weren't read are eliminated
+	- `max_size`: limit the amount of entries
+	- `typed`: `typed=true` means that `f(1)` and `f(1.0)` are stored separately
+
+## singledispatch
+
+- Imitate the **overloading** from **Java**
+	- A function implementation can vary, and the called version depends on only from the first type parameter
+- Use `collections.abc` and `protocols` for the types because they are better for future modification and `virtual
+  classes`. In other words, **avoid concrete types**
+- [[rugi-singledispatch-implementation]]#
+
+### Reference
+
+- Ramalho, 2022, p320-329