diff --git a/docs/newsletter/2024_w31.md b/docs/newsletter/2024_w31.md new file mode 100644 index 00000000000..44941cde73f --- /dev/null +++ b/docs/newsletter/2024_w31.md @@ -0,0 +1,1868 @@ +# [Activism](antiracism.md) + +* New: Añadir el vídeo del racismo no se sostiene. + + [El racismo no se sostiene](https://youtube.com/shorts/5Y7novO2t_c?si=dqMGW4ALFLoXZiw3) + +## [Free Knowledge](free_knowledge.md) + +* Correction: Update the way of seeding ill knowledge torrents. + + A good way to contribute is by seeding the ill torrents. You can [generate a list of torrents that need seeding](https://annas-archive.org/torrents#generate_torrent_list) up to a limit in TB. If you follow this path, take care of IP leaking, they're + +## [Conference organisation](conference_organisation.md) + +* New: Software to manage the conference. + + There are some open source software that can make your life easier when hosting a conference: + + - [Frab](https://frab.github.io/frab/) + - [Pretalx](https://pretalx.com/p/about/) + - [Wafer](https://wafer.readthedocs.io/en/latest/) + + In addition to the management of talks from the call for papers till the event itself it can help the users visualise the talks schedule with [EventFahrplan](https://github.com/EventFahrplan/EventFahrplan?tab=readme-ov-file) which is what's used in the ChaosComputerClub congress. + + If you also want to coordinate helpers and shifts take a look to [Engelsystem](https://engelsystem.de/en) + +# Life Management + +## [Time management](time_management.md) + +* New: Keep on summing up Oliver Burkeman book. + + **Efficiency doesn't necessarily give you more time** + + We're eager to believe the promises of time management frameworks (like [GTD](gtd.md)) that if you improve your efficiency you'll get more time to enjoy your life. If you follow the right time management system, build the right habits, and apply sufficient self-discipline, you will win the struggle with time. + + Reality then kicks in you never win the struggle and only feel more stressed and unhappy. You realize that all the time you've saved is automatically filled up by more things to do in a never ending feedback loop. Time feels like an unstoppable conveyor belt, bringing us new actions as fast as we can dispatch the old ones; and becoming more efficient just seems to cause the belt to speed up. Or else, eventually, to break down. It's true that you get more done, and yet, paradoxically, you only feel busier, more anxious and somehow emptier as a result. + + It get's even worse because [importance is relative](time_management.md#importance-is-relative) and you may fall into [efficiency traps](time_management.md#be-mindful-of-the-efficiency-trap). + + **Heal yourself from FOMO** + + Another problem that FOMO brings us is that it leads us to lives where you "truly lived" only if you've lived all the experiences you could live. This leads to a frustrating life as the world has infinite of them, so getting a handful of them under your belt brings you no closer to a sense of having feasted on life's possibilities. You lead yourself in another [efficiency trap](#be-mindful-of-the-efficiency-trap) where the more you experience the more additional wonderful experiences you sarta to feel you could have on top of all those you've already had, with the result that the feeling of existential overwhelm gets worse. To fight this existential overwhelm you can resist the urge to consume more and more experiences and embrace the idea that you're going to miss most of them. You'll then be able to focus on fully enjoying the tiny slice of experiences you actually do have time for. + + This FOMO fever is normal given the facts that we're more conscious of the limits of our time (after deterring the after life), the increase of choices that the world has brought us, and the internet amplifier. + + **You do what you can do** + + It's usual to feel as though you absolutely must do more than you can do. We live overwhelmed in a constant anxiety of fearing, or knowing for certain, that the actions we want to carry out won't fit on our available time. It looks like this feeling arises on every step of the economic ladder (shown in the works of Daniel Markovits). + + The thing is that the idea in itself doesn't make any sense. You can't do more than you can do even if you must. If you truly don't have time for everything you want to do, or feel you ought to do, or that others are badgering you to do, then, well, you don't have time, no matter how grave the consequences of failing to do it all might prove to be. So technically it's irrational to feel troubled by an overwhelming to-do list. You'll do what you can, you won't do what you can't, and the tyrannical inner voice insisting that you must do everything is simply mistaken. We rarely stop to consider things so rationally, though, because that would mean confronting the painful truth of our limitations. We would be forced to acknowledge that there are hard choices to be made: which balls to let drop, which people to disappoint, which ambitions to abandon, which roles to fail at... Instead, in an attempt to avoid these unpleasant truths, we deploy the strategy that dominates most conventional advice on how to deal with busyness: we tell ourselves we'll just have to find a way to do more. So to address our busyness we're making ourselves busier still. + + **Importance is relative** + + The problem here is that you'll never be able to make time for everything that feels important. A similar mindset of the section [Efficiency doesn't give you more time](#efficiency-doesnt-give-you-more-time) can be applied. The reason isn't that you haven't yet discovered the right time management tricks, or applied sufficient effort, or that you're generally useless. It's that the underlying assumption is unwarranted: there's no reason to believe you'll make time for everything that matters simply by getting more done. For a start, what "matters" is subjective, so you've no grounds for assuming that there will be time for everything that you, or anyone else deems important. But the other exasperating issue is that if you succeed in fitting more in, you'll find the goalposts start to shift: more things will begin to seem important, meaningful or obligatory. Acquire a reputation for doing your work at amazing speed, and you'll be given more of it. An example of this is gathered in Ruth Schwartz's book More work for mother, where it shows that when washing machines and vacuum cleaners appeared no time was saved at all, because society's standards of cleanliness rose to offset the benefits. What needs doing expands so as to fill the time available for its completion. + + **Be mindful of the efficiency trap** + + Sometimes improving your efficiency may lead you to a worse scenario ("efficiency trap") where you won't generally result in the feeling of having "enough time", because, all else being equal, the demands will increase to offset any benefits. Far from getting things done, you'll be creating new things to do. A clear example of this is email management. Every time you reply to an email, there's a good chance of provoking a reply to that email, which itself may require another reply, and so on and so on. At the same time, you'll become known as someone who responds promptly to email, so more people will consider it worth their while to message you to begin with. So it's not simply that you never get though your email; it's that the process of "getting through your email" actually generates more email. + + For most of us, most of the time, it isn't feasible to avoid the efficiency trap altogether, but you can stop believing you'll ever solve the challenge of busyness by cramming more in, because that just makes matters worse. And once you stop investing in the idea that you might one day achieve peace of mind that way, it becomes easier to find peace of mind in the present, in the midst of overwhelming demands, because you're no longer making your peace of mind dependent on dealing with all the demands. Once you stop believing that it might somehow be possible to avoid hard choices about time, it gets easier to make better ones. + + If you also have the knowledge of the existence of the efficiency traps you may detect them and try to get the benefits without the penalties. + + **Do the important stuff** + + The worst aspect of the trap is that it's also a matter of quality. The harder you struggle to fit everything in, the more of your time you'll find yourself spending on the least meaningful things. This is because the more firmly you believe it ought to be possible to find time for everything, the less pressure you'll feel to ask whether any given activity sis the best use of a portion of your time. Each time something new shows up, you'll be strongly biased in favor of accepting it, because you'll assume you needn't sacrifice any other tasks or opportunities in order to make space for it. Soon your life will be automatically filled with not just more things but with more trivial or tedious things. + + The important stuff gets postponed because such tasks need your full focus, which means to wait until you have a good chunk of free time and fewer small-but-urgent tasks tugging at your attention. So you spend your energy into clearing the decks, cranking through the smaller stuff to get it out of the way, only to discover that doing so takes the whole day, that the decks are filled up again overnight and that the moment for doing the important stuff never arrives. One can waste years this way, systematically postponing precisely the things one cares the most. + + What's needed in these situations is to resist the urges of being on top of everything and learn to live with the anxiety of feeling overwhelmed without automatically responding by trying to fit more in. Instead of clearing the decks, decline to do so, focusing instead on what's truly of greatest consequence while tolerating the discomfort of knowing that, as you do so, the decks will be filling up further, with emails and errands and other to-dos, many of which you may never get around to at all. + + You'll sometimes still decide to drive yourself hard in an effort to squeeze more in, when circumstances absolutely require it. But that won't be your default mode, because you'll no longer be operating under the illusion of one day making time for everything. + + **Evaluate what you miss when you increase your efficiency** + + Part of the benefits of efficiency is that you free yourself from tedious experiences, the side effect is that some times we're not conscious of being removing experiences that we valued. So even if everything runs more smoothly, smoothness is a dubious virtue, since it's often the unsmoothed textures of life that makes them livable, helping nurture the relationships that are crucial for mental and physical health, and for the resilience of our communities. For example if you buy online the groceries you miss the chance to regularly meet with your neighbours at your local grocery store. + + Convenience makes things easy, but without regard to whether easiness is truly what's most valuable in any given context. When you render the process more convenient you drain it of its meaning. The effect of convenience isn't just that the given activity starts to feel less valuable, but that we stop engaging in certain valuable activities altogether, in favour of more convenient ones. Because you can stay home, order food online, and watch sitcoms on a streaming service, you find yourself doing so although you might be perfectly aware that you'd have had a better time if you had met with your friends. + + Meanwhile, those aspects of life that resist being made to run more smoothly start to seem repellent. When you can skip the line and buy concert tickets on your phone, waiting in line to vote in an election is irritating. As convenience colonizes everyday life, activities gradually sort themselves into two types: the kind that are now far more convenient, but that feel empty or out of sync with our true preferences; and the kind that now seem intensely annoying because of how inconvenient they remain. Resisting all this is difficult because the Capital is winning this discourse and you'll have more pressure from your environment to stay convenient. + +### [Life planning](time_management_abstraction_levels.md) + +* Correction: Tweak the month planning. + + Add the next steps: + + - Clean your agenda and get an feeling of the busyness of the month: + - Open the orgmode month view agenda and clean it + - Read the rest of your calendars + + Then reorder the objectives in order of priority. Try to have at least one objective that improves your life. + + - For each of your month and trimester objectives: + - Decide whether it makes sense to address it this month. If not, mark it as inactive + - Create a clear plan of action for this month on that objective. + - Reorder the projects as needed + - Mark as INACTIVE the ones that you don't feel need to be focused on this month. + + - Refine the roadmap of each of the selected areas (change this to the trimestral planning) + - Select at least one coding project in case you enter in programming mode + - Clean your mobile browser tabs + +* New: [Debug doesn't go up in the jump list.](orgmode.md#-doesn't-go-up-in-the-jump-list) + + It's because [ is a synonym of ](https://github.com/neovim/neovim/issues/5916), and `org_cycle` is [mapped by default as ](https://github.com/nvim-orgmode/orgmode/blob/c0584ec5fbe472ad7e7556bc97746b09aa7b8221/lua/orgmode/config/defaults.lua#L146) + If you're used to use `zc` then you can disable the `org_cycle` by setting the mapping `org_cycle = ""`. + +* New: [Python libraries.](orgmode.md#python-libraries) + + **[org-rw](https://code.codigoparallevar.com/kenkeiras/org-rw)** + + `org-rw` is a library designed to handle Org-mode files, offering the ability to modify data and save it back to the disk. + + - **Pros**: + - Allows modification of data and saving it back to the disk + - Includes tests to ensure functionality + + - **Cons**: + - Documentation is lacking, making it harder to understand and use + - The code structure is complex and difficult to read + - Uses `unittest` instead of `pytest`, which some developers may prefer + - Tests are not easy to read + - Last commit was made five months ago, indicating potential inactivity + - [Not very popular]( https://github.com/kenkeiras/org-rw), with only one contributor, three stars, and no forks + + **[orgparse](https://github.com/karlicoss/orgparse)** + + `orgparse` is a more popular library for parsing Org-mode files, with better community support and more contributors. However, it has significant limitations in terms of editing and saving changes. + + - **Pros**: + - More popular with 13 contributors, 43 forks, and 366 stars + - Includes tests to ensure functionality + - Provides some documentation, available [here](https://orgparse.readthedocs.io/en/latest/) + + - **Cons**: + - Documentation is not very comprehensive + - Cannot write back to Org-mode files, limiting its usefulness for editing content + - The author suggests using [inorganic](https://github.com/karlicoss/inorganic) to convert Org-mode entities to text, with examples available in doctests and the [orger](https://github.com/karlicoss/orger) library. + - `inorganic` is not popular, with one contributor, four forks, 24 stars, and no updates in five years + - The library is only 200 lines of code + - The `ast` is geared towards single-pass document reading. While it is possible to modify the document object tree, writing back changes is more complicated and not a common use case for the author. + + **[Tree-sitter](https://tree-sitter.github.io/tree-sitter/)** + + Tree-sitter is a powerful parser generator tool and incremental parsing library. It can build a concrete syntax tree for a source file and efficiently update the syntax tree as the source file is edited. + + - **Pros**: + - General enough to parse any programming language + - Fast enough to parse on every keystroke in a text editor + - Robust enough to provide useful results even in the presence of syntax errors + - Dependency-free, with a runtime library written in pure C + - Supports multiple languages through community-maintained parsers + - Used by Neovim, indicating its reliability and effectiveness + - Provides good documentation, available [here](https://tree-sitter.github.io/tree-sitter/using-parsers) + - Python library, [py-tree-sitter](https://github.com/tree-sitter/py-tree-sitter), simplifies the installation process + + - **Cons**: + - Requires installation of Tree-sitter and the Org-mode language parser separately + - The Python library does not handle the Org-mode language parser directly + + To get a better grasp of Tree-sitter you can check their talks: + + - [Strange Loop 2018](https://www.thestrangeloop.com/2018/tree-sitter---a-new-parsing-system-for-programming-tools.html) + - [FOSDEM 2018](https://www.youtube.com/watch?v=0CGzC_iss-8) + - [Github Universe 2017](https://www.youtube.com/watch?v=a1rC79DHpmY). + + **[lazyblorg orgparser.py](https://github.com/novoid/lazyblorg/blob/master/lib/orgparser.py)** + + `lazyblorg orgparser.py` is another tool for working with Org-mode files. However, I didn't look at it. + +* Correction: [Tweak area concept.](time_management_abstraction_levels.md#area) + + Model a group of projects that follow the same interest, roles or accountabilities. These are not things to finish but rather to use as criteria for analyzing, defining a specific aspect of your life and to prioritize its projects to reach a higher outcome. We'll use areas to maintain balance and sustainability on our responsibilities as we operate in the world. Areas' titles don't contain verbs as they don't model actions. An example of areas can be *health*, *travels* or *economy*. + + To filter the projects by area I set an area tag that propagates downstream. To find the area documents easily I add a section in the `index.org` of the documentation repository. For example: + +### [vdirsyncer](vdirsyncer.md) + +* New: [Troubleshoot Database is locked.](vdirsyncer.md#database-is-locked) + + First try to kill all stray vdirsyncer processes, if that doesn't work check for more solutions in [this issue](https://github.com/pimutils/vdirsyncer/issues/720) + +### [Roadmap Adjustment](roadmap_adjustment.md) + +* New: [Trimester review.](roadmap_adjustment.md#trimester-review) + + The objectives of the trimester review are: + + - Identify the areas to focus on for the trimester + - Identify the tactics you want to use on those areas. + - Review the previous trimester tactics + + The objectives are not: + + - To review what you've done or why you didn't get there. + + **When to do the trimester reviews** + + As with [personal integrity review](#personal-integrity-review), it's interesting to do analysis at representative moments. It gives it an emotional weight. You can for example use the solstices or my personal version of the solstices: + + - Spring analysis (1st of March): For me the spring is the real start of the year, it's when life explodes after the stillness of the winter. The sun starts to set later enough so that you have light in the afternoons, the climate gets warmer thus inviting you to be more outside, the nature is blooming new leaves and flowers. It is then a moment to build new projects and set the current year on track. + - Summer analysis (1st of June): I hate heat, so summer is a moment of retreat. Everyone temporarily stop their lives, we go on holidays and all social projects slow their pace. Even the news have even less interesting things to report. It's so hot outside that some of us seek the cold refuge of home or remote holiday places. Days are long and people love to hang out till late, so usually you wake up later, thus having less time to actually do stuff. Even in the moments when you are alone the heat drains your energy to be productive. It is then a moment to relax and gather forces for the next trimester. It's also perfect to develop *easy* and *chill* personal projects that have been forgotten in a drawer. Lower your expectations and just flow with what your body asks you. + - Autumn analysis (1st of September): September it's another key moment for many people. We have it hardcoded in our life since we were children as it was the start of school. People feel energized after the summer holidays and are eager to get back to their lives and stopped projects. You're already 6 months into the year, so it's a good moment to review your year plan and decide how you want to invest your energy reserves. + - Winter analysis (1st of December): December is the cue that the year is coming to an end. The days grow shorter and colder, they basically invite you to enjoy a cup of tea under a blanket. It is then a good time to get into your cave and do an introspection analysis on the whole year and prepare the ground for the coming year. Some of the goals of this season are: + - Think everything you need to guarantee a good, solid and powerful spring start. + - Do the year review to adjust your principles. + + The year is then divided in two sets of an expansion trimester and a retreat one. We can use this information to adjust our life plan accordingly. In the expansion trimester we could invest more energies in the planning, and in the retreat ones we can do more throughout reviews. + + **Listen to your desires** + + The trimester review requires an analysis that doesn't fill in a day session. It requires slow thinking over some time. So I'm creating a task 10 days before the actual review to start thinking about the next trimester. Whether it's ideas, plans, desires, objectives, values, or principles. + + Is useful for that document to be available wherever you go, so that in any spare time you can pop it up and continue with the train of thought. + + Doing the reflection without seeing your life path prevents you from being tainted by it, thus representing the real you of right now. + + On the day to actually do the review, follow the steps of the [Month review prepare](#month-prepare) adjusting them to the trimester case. + + **Answer some meaningful guided questions** + + To be done, until then you can read chapters 13, 14 and the epilogue of the book Four thousand weeks by Oliver Burkman. + + **Refactor your gathered thoughts** + + If you've followed the prepare steps, you've already been making up your mind on what do you want the next trimester to look like. Now it's the time to refine those thoughts. + + In your roadmap document add a new section for the incoming trimester similar to: + + ```orgmode + * Roadmap + ** 2024 + *** Summer 2024 + **** Essential intent + **** Trimester analysis + **** Trimester objectives + ***** TODO Objective 1 + ****** TODO SubObjective 1 + ``` + Go *one by one* (don't peek!) of your gathered items and translate them in the next sections: + + - `Trimester analysis`: A text with as many paragraphs as you need to order your thoughts + - `Trimester objectives`: These can be concrete emotional projects you want to carry through. + - `Essential intent`: This is the main headline of your trimester, probably you won't be able to define it until the last parts of the review process. It should be concrete and emotional too, it's going to be the idea that gives you strength on your weak moments and your guide to decide which projects to do and which not to. + + Don't be too concerned on the format of the content of the objectives, this is the first draft, and we'll refine it through the planning. + +## Content Management + +### [Jellyfin](jellyfin.md) + +* New: [Troubleshoot pSystem.InvalidOperationException: There is an error in XML document (0, 0).](jellyfin.md#system.invalidoperationexception:-there-is-an-error-in-xml-document-0,-0) + + This may happen if you run out of disk and some xml file in the jellyfin data directory becomes empty. The solution is to restore that file from backup. + +### [Mediatracker](mediatracker.md) + +* New: [Add missing books.](mediatracker.md#add-missing-books) + + - Register an account in openlibrary.com + - Add the book + - Then add it to mediatracker + +## Knowledge Management + +### [Aleph](aleph.md) + +* New: [Debug ingestion errors.](aleph.md#debug-ingestion-errors) + + Assuming that you've [set up Loki to ingest your logs](https://github.com/alephdata/aleph/issues/2124) I've so far encountered the next ingest issues: + + - `Cannot open image data using Pillow: broken data stream when reading image files`: The log trace that has this message also contains a field `trace_id` which identifies the ingestion process. With that `trace_id` you can get the first log trace with the field `logger = "ingestors.manager"` which will contain the file path in the `message` field. Something similar to `Ingestor []` + - A traceback with the next string `Failed to process: Could not extract PDF file: FileDataError('cannot open broken document')`: This log trace has the file path in the `message` field. Something similar to `[] Failed to process: Could not extract PDF file: FileDataError('cannot open broken document')` + + I thought of making a [python script to automate the files that triggered an error](loki.md#interact-with-loki-through-python), but in the end I extracted the file names manually as they weren't many. + + Once you have the files that triggered the errors, the best way to handle them is to delete them from your investigation and ingest them again. + +* New: [Add support channel.](aleph.md#references) + + [Support chat](https://alephdata.slack.com) + +## Torrent management + +### [qBittorrent](qbittorrent.md) + +* New: [Troubleshoot Trackers stuck on Updating.](qbittorrent.md#trackers-stuck-on-updating) + + Sometimes the issue comes from an improvable configuration. In advanced: + + - Ensure that there are enough [Max concurrent http announces](https://github.com/qbittorrent/qBittorrent/issues/15744): I changed from 50 to 500 + - [Select the correct interface and Optional IP address to bind to](https://github.com/qbittorrent/qBittorrent/issues/14453). In my case I selected `tun0` as I'm using a vpn and `All IPv4 addresses` as I don't use IPv6. + +# Health + +## [Teeth](teeth.md) + +* New: Suggestion on how to choose the toothpaste to buy. + + When choosing a toothpaste choose the one that has a higher percent of fluoride. + +# Coding + +## Languages + +### [Bash snippets](bash_snippets.md) + +* New: [Fix docker error: KeyError ContainerConfig.](bash_snippets.md#fix-docker-error:-keyerror-containerconfig) + + You need to run `docker-compose down` and then up again. + +* New: [Set static ip with nmcli.](bash_snippets.md#set-static-ip-with-nmcli) + + ```bash + nmcli con mod "your-ssid" ipv4.addresses + ipv4.method "manual" \ + ipv4.addresses "your_desired_ip" \ + ipv4.gateway "your_desired_gateway" \ + ipv4.dns "1.1.1.1,2.2.2.2" \ + ipv4.routes "192.168.32.0 0.0.0.0" \ + ``` + + The last one is to be able to connect to your LAN, change the value accordingly. + +* New: [Fix unbound variable error.](bash_snippets.md#fix-unbound-variable-error) + + You can check if the variable is set and non-empty with: + ```bash + [ -n "${myvariable-}" ] + ``` + +* New: [Compare two semantic versions with sort.](bash_snippets.md#with-sort) + + If you want to make it work in non-Debian based systems you can use `sort -V -C` + + ```bash + printf "2.0.0\n2.1.0\n" | sort -V -C # Return code 0 + printf "2.2.0\n2.1.0\n" | sort -V -C # Return code 1 + ``` + +### [Graphql](graphql.md) + +* New: Introduce GraphQL. + + [GraphQL](https://graphql.org/) is a query language for APIs and a runtime for fulfilling those queries with your existing data. GraphQL provides a complete and understandable description of the data in your API, gives clients the power to ask for exactly what they need and nothing more, makes it easier to evolve APIs over time, and enables powerful developer tools. + + To use it with python you can use [Ariadne](https://ariadnegraphql.org/) ([source](https://github.com/mirumee/ariadne)) + +### [Python Snippets](python_snippets.md) + +* New: [Compare file and directories.](python_snippets.md#compare-file-and-directories) + + The filecmp module defines functions to compare files and directories, with various optional time/correctness trade-offs. For comparing files, see also the difflib module. + + ```python + from filecmp import dircmp + + def print_diff_files(dcmp): + for name in dcmp.diff_files: + print("diff_file %s found in %s and %s" % (name, dcmp.left, dcmp.right)) + for sub_dcmp in dcmp.subdirs.values(): + print_diff_files(sub_dcmp) + dcmp = dircmp('dir1', 'dir2') + print_diff_files(dcmp) + ``` + +## Coding tools + +### [Singer](vim.md) + +* New: Introduce singer. + + [Singer](https://www.singer.io/) is an open-source standard for writing scripts that move data. + + It describes how data extraction scripts—called “taps” —and data loading scripts—called “targets”— should communicate, allowing them to be used in any combination to move data from any source to any destination. Send data between databases, web APIs, files, queues, and just about anything else you can think of. + + It has many "taps" and "targets" that can help you interact with third party tools without needing to write the code. + + **References** + - [Home](https://www.singer.io/) + +* New: [ is not well mapped.](vim.md#-is-not-well-mapped) + + It's because `` is a synonym of ``. + +### [Coding with AI](kestra.md) + +* New: Introduce ai coding prompts. + + These are some useful AI prompts to help you while you code: + + - create a function with type hints and docstring using google style called { } that { } + - create the tests for the function { } adding type hints and following the AAA style where the Act section is represented contains a returns = (thing to test) line or if the function to test doesn't return any value append an # act comment at the end of the line. Use paragraphs to separate the AAA blocks and don't add comments inside the tests for the sections + + If you use [espanso](espanso.md) you can simplify the filling up of these prompts on the AI chats. For example: + + ```yaml + --- + matches: + - trigger: :function + form: | + Create a function with type hints and docstring using google style called [[name]] that: + [[text]] + form_fields: + text: + multiline: true + - trigger: :tweak + form: | + Tweak the next code: + [[code]] + + So that: + + [[text]] + form_fields: + text: + multiline: true + code: + multiline: true + - trigger: :test + form: | + create the tests for the function: + [[text]] + + Following the next guidelines: + + - Add type hints + - Follow the AAA style + - In the Act section if the function to test returns a value always name that variable returns. If the function to test doesn't return any value append an # act comment at the end of the line. + - Use paragraphs to separate the AAA blocks and don't add comments like # Arrange or # Act or # Act/Assert or # Assert + + form_fields: + text: + multiline: true + - trigger: :refactor + form: | + Refactor the next code + [[code]] + with the next conditions + [[conditions]] + form_fields: + code: + multiline: true + conditions: + multiline: true + ``` + +* New: Introduce Kestra. + + [Kestra](https://kestra.io/) is an [open-source orchestrator](data_orchestrator.md) designed to bring Infrastructure as Code (IaC) best practices to all workflows — from those orchestrating mission-critical operations, business processes, and data pipelines to simple Zapier-style automation. Built with an API-first philosophy, Kestra enables users to define and manage data pipelines through a simple YAML configuration file. This approach frees you from being tied to a specific client implementation, allowing for greater flexibility and easier integration with various tools and services. + + Look at this [4 minute video](https://www.youtube.com/watch?v=h-P0eK2xN58) for a visual introduction + + **References** + - [Docs](https://kestra.io/docs/getting-started) + - [Home](https://kestra.io/) + - [4 minute introduction video](https://www.youtube.com/watch?v=h-P0eK2xN58) + +### [memorious](memorious.md) + +* New: Introduce memorious. + + [Memorious](https://github.com/alephdata/memorious) is a light-weight web scraping toolkit. It supports scrapers that collect structured or un-structured data. This includes the following use cases: + + - Make crawlers modular and simple tasks re-usable + - Provide utility functions to do common tasks such as data storage, HTTP session management + - Integrate crawlers with the Aleph and FollowTheMoney ecosystem + + **References** + + - [Memorious](https://github.com/alephdata/memorious) + +### [Data orchestrators](data_orchestrators.md) + +* New: Introduce data orchestrators. + + Data orchestration is the process of moving siloed data from multiple storage locations into a centralized repository where it can then be combined, cleaned, and enriched for activation. + + Data orchestrators are web applications that make this process easy. The most popular right now are: + + - Apache Airflow + - [Kestra](#kestra) + - Prefect + + There are several comparison pages: + + - [Geek Culture comparison](https://medium.com/geekculture/airflow-vs-prefect-vs-kestra-which-is-best-for-building-advanced-data-pipelines-40cfbddf9697) + - [Kestra's comparison to Airflow](https://kestra.io/vs/airflow) + - [Kestra's comparison to Prefect](https://kestra.io/vs/prefect) + + When looking at the return on investment when choosing an orchestration tool, there are several points to consider: + + - Time of installation/maintenance + - Time to write pipeline + - Time to execute (performance) + + **[Kestra](kestra.md)** + + Pros: + + - Easier to write pipelines + - Nice looking web UI + - It has a [terraform provider](https://kestra.io/docs/getting-started/terraform) + - [Prometheus and grafana integration](https://kestra.io/docs/how-to-guides/monitoring) + + Cons: + + - Built in Java, so extending it might be difficult + - [Plugins are made in Java](https://kestra.io/docs/developer-guide/plugins) + + Kestra offers a higher ROI globally compared to Airflow: + + - Installing Kestra is easier than Airflow; it doesn’t require Python dependencies, and it comes with a ready-to-use docker-compose file using few services and without the need to understand what’s an executor to run task in parallel. + - Creating pipelines with Kestra is simple, thanks to its syntax. You don’t need knowledge of a specific programming language because Kestra is designed to be agnostic. The declarative YAML design makes Kestra flows more readable compared to Airflow’s DAG equivalent, allowing developers to significantly reduce development time. + - In this benchmark, Kestra demonstrates better execution time than Airflow under any configuration setup. + +### [Scrapers](morph_io.md) + +* New: Introduce morph.io. + + [morph.io](https://morph.io/) is a web service that runs your scrapers for you. + + Write your scraper in the language you know and love, push your code to GitHub, and they take care of the boring bits. Things like running your scraper regularly, alerting you if there's a problem, storing your data, and making your data available for download or through a super-simple API. + + To sign in you'll need a GitHub account. This is where your scraper code is stored. + + The data is stored in an sqlite + + **Usage limits** + + Right now there are very few limits. They are trusting you that you won't abuse this. + + However, they do impose a couple of hard limits on running scrapers so they don't take up too many resources + + - max 512 MB memory + - max 24 hours run time for a single run + + If a scraper runs out of memory or runs too long it will get killed automatically. + + There's also a soft limit: + + - max 10,000 lines of log output + + If a scraper generates more than 10,000 lines of log output the scraper will continue running uninterrupted. You just won't see any more output than that. To avoid this happening simply print less stuff to the screen. + + Note that they are keeping track of the amount of cpu time (and a whole bunch of other metrics) that you and your scrapers are using. So, if they do find that you are using too much they reserve the right to kick you out. In reality first they'll ask you nicely to stop. + + **References** + + - [Docs](https://morph.io/documentation) + - [Home](https://morph.io/) + +### [Espanso](espanso.md) + +* New: Introduce espanso. + + [Espanso](https://github.com/espanso/espanso) is a cross-platform Text Expander written in Rust. + + A text expander is a program that detects when you type a specific keyword and replaces it with something else. This is useful in many ways: + + - Save a lot of typing, expanding common sentences or fixing common typos. + - Create system-wide code snippets. + - Execute custom scripts + - Use emojis like a pro. + + **[Installation](https://espanso.org/docs/install/linux/)** + Espanso ships with a .deb package, making the installation convenient on Debian-based systems. + + Start by downloading the package by running the following command inside a terminal: + + ```bash + wget https://github.com/federico-terzi/espanso/releases/download/v2.2.1/espanso-debian-x11-amd64.deb + ``` + + You can now install the package using: + + ```bash + sudo apt install ./espanso-debian-x11-amd64.deb + ``` + + From now on, you should have the `espanso` command available in the terminal (you can verify by running `espanso --version`). + + At this point, you are ready to use `espanso` by registering it first as a Systemd service and then starting it with: + + ```bash + espanso service register + ``` + + Start espanso + + ```bash + espanso start + ``` + + Espanso ships with very few built-in matches to give you the maximum flexibility, but you can expand its capabilities in two ways: creating your own custom matches or [installing packages](#using-packages). + + **[Configuration](https://espanso.org/docs/get-started/#configuration)** + + Your configuration lives at `~/.config/espanso`. A quick way to find the path of your configuration folder is by using the following command `espanso path`. + + - The files contained in the `match` directory define what Espanso should do. In other words, this is where you should specify all the custom snippets and actions (aka Matches). The `match/base.yml` file is where you might want to start adding your matches. + - The files contained in the `config` directory define how Espanso should perform its expansions. In other words, this is were you should specify all Espanso's parameters and options. The `config/default.yml` file defines the options that will be applied to all applications by default, unless an app-specific configuration is present for the current app. + + **[Using packages](https://espanso.org/docs/get-started/#understanding-packages)** + + Custom matches are great, but sometimes it can be tedious to define them for every common operation, especially when you want to share them with other people. + + Espanso offers an easy way to share and reuse matches with other people, packages. In fact, they are so important that Espanso includes a built-in package manager and a store, the [Espanso Hub](https://hub.espanso.org/). + + **[Installing a package](https://espanso.org/docs/get-started/#installing-a-package)** + + Get the id of the package from the [Espanso Hub](https://hub.espanso.org/) and then run `espanso install <>`. + + Of all the packages, I've found the next ones the most useful: + + - [typofixer-en](https://hub.espanso.org/typofixer-en) + - [typofixer-es](https://hub.espanso.org/typofixer-es) + - [misspell-en-uk](https://hub.espanso.org/misspell-en-uk) + + **Overwriting the snippets of a package** + + For example the `typofixer-en` replaces `si` to `is`, although `si` is a valid spanish word. To override the fix you can create your own file on `~/.config/espanso/match/typofix_overwrite.yml` with the next content: + + ```yaml + matches: + # Simple text replacement + - trigger: "si" + replace: "si" + ``` + + **[Creating a package](https://espanso.org/docs/packages/creating-a-package/)** + + **Auto-restart on config changes** + + Set `auto_restart: true` on `~/.config/espanso/config/default.yml`. + + **[Changing the search bar shortcut](https://espanso.org/docs/configuration/options/#customizing-the-search-bar)** + + If the default search bar shortcut conflicts with your i3 configuration set it with: + + ```yaml + search_shortcut: CTRL+SHIFT+e + ``` + + **[Hiding the notifications](https://espanso.org/docs/configuration/options/#hiding-the-notifications)** + + You can hide the notifications by adding the following option to your `$CONFIG/config/default.yml` config: + + ```yaml + show_notifications: false + ``` + + **Usage** + + Just type and you'll see the text expanded. + + You can use the search bar if you don't remember your snippets. + + **References** + - [Code](https://github.com/espanso/espanso) + - [Docs](https://espanso.org/docs/get-started/) + +## Generic Coding Practices + +### [Conventional comments](conventional_comments.md) + +* New: Introduce conventional comments. + + [Conventional comments](https://conventionalcomments.org/) is the practice to use a specific format in the review comments to express your intent and tone more clearly. It's strongly inspired by [semantic versioning](semantic_versioning.md). + + Let's take the next comment: + + ``` + This is not worded correctly. + ``` + + Adding labels you can tell the difference on your intent: + + ``` + **suggestion:** This is not worded correctly. + ``` + Or + ``` + **issue (non-blocking):** This is not worded correctly. + ``` + + Labels also prompt the reviewer to give more **actionable** comments. + + ``` + **suggestion:** This is not worded correctly. + + Can we change this to match the wording of the marketing page? + ``` + + Labeling comments encourages collaboration and saves **hours** of undercommunication and misunderstandings. They are also parseable by machines! + + **Format** + + Adhering to a consistent format improves reader's expectations and machine readability. + Here's the format we propose: + ``` +