Skip to content

Commit

Permalink
V2.0.0 JANUARY.1 2024
Browse files Browse the repository at this point in the history
  • Loading branch information
@Erik172
Erik172 committed Jan 6, 2024
2 parents 610fe4e + 456b6da commit 32cbd90
Show file tree
Hide file tree
Showing 52 changed files with 614,878 additions and 1,547,116 deletions.
44 changes: 19 additions & 25 deletions .github/ISSUE_TEMPLATE/bug_report.md
View file
Original file line number Diff line number Diff line change
@@ -1,38 +1,32 @@
---
name: Bug report
about: Create a report to help us improve
name: Scraper Bug Report 🐞
about: Report issues or bugs related to the scraper tool
title: ''
labels: ''
labels: 'bug'
assignees: ''

---

**Describe the bug**
A clear and concise description of what the bug is.
**Describe the Bug**
A clear and concise description of the issue encountered with the scraper.

**To Reproduce**
Steps to reproduce the behavior:
**Steps to Reproduce**
Please provide steps to replicate the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

**Expected behavior**
A clear and concise description of what you expected to happen.
4. See the error occurring with the scraper.

**Screenshots**
If applicable, add screenshots to help explain your problem.
**Expected Behavior**
A clear and concise description of what you expected the scraper to do.

**Desktop (please complete the following information):**
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
**Screenshots or Code Snippets**
If applicable, add screenshots or code snippets to illustrate the problem.

**Smartphone (please complete the following information):**
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]
**Environment Details**
Please complete the following information:
- OS: [e.g. Windows, macOS, Linux]
- Browser (if applicable): [e.g. Chrome, Firefox]
- Scraper Version: [e.g. 1.0.0]

**Additional context**
Add any other context about the problem here.
**Additional Context**
Add any other context about the issue encountered with the scraper.
14 changes: 7 additions & 7 deletions .github/ISSUE_TEMPLATE/feature_request.md
View file
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
---
name: Feature request
about: Suggest an idea for this project
name: Feature Request 🚀
about: Suggest an idea or enhancement related to the scraper or data
title: ''
labels: ''
labels: 'enhancement'
assignees: ''

---

**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
A clear and concise description of the problem or limitation you're facing with the scraper or data.

**Describe the solution you'd like**
A clear and concise description of what you want to happen.
A clear and concise description of the feature or enhancement you'd like to see implemented in the scraper or data.

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
Have you thought about any alternative solutions or features? Describe them briefly.

**Additional context**
Add any other context or screenshots about the feature request here.
Add any other context, examples, or screenshots about the feature request here.
41 changes: 20 additions & 21 deletions .github/ISSUE_TEMPLATE/issue-template.md
View file
Original file line number Diff line number Diff line change
@@ -1,48 +1,47 @@
---
name: issue template
about: Proporcionamos una descripción breve del problema o tarea y las tareas necesarias
para resolverlo.
name: Scraper or Data Issue 🐛
about: Describe an issue or task related to the scraper or data and the necessary steps to resolve it.
title: ''
labels: ''
assignees: Erik172

---

## Título del Issue
## Issue Title

Breve descripción del problema o tarea que se debe abordar.
Brief description of the problem or task related to the scraper or data that needs addressing.

### Descripción
### Description

Proporcione una descripción más detallada del problema o tarea.
Provide a detailed description of the issue or task related to the scraper or data.

### Pasos para reproducir el problema
### Steps to Reproduce (If Applicable)

Si se trata de un problema, proporcione los pasos específicos para reproducir el problema:
If it's a problem, provide specific steps to reproduce it:

1.
2.
3.

### Comportamiento esperado
### Expected Behavior

Si se trata de un problema, describa el comportamiento esperado.
If it's a problem, describe the expected behavior related to the scraper or data.

### Comportamiento actual
### Current Behavior

Si se trata de un problema, describa el comportamiento actual.
If it's a problem, describe the current behavior related to the scraper or data.

### Capturas de pantalla
### Screenshots or Code Snippets

Si es posible, proporcione capturas de pantalla que muestren el problema o la tarea.
If possible, provide screenshots or code snippets illustrating the problem or task.

### Información adicional
### Additional Information

Proporcione cualquier otra información relevante que pueda ayudar a abordar el problema o tarea.
Provide any other relevant information that might help in addressing the issue or task related to the scraper or data.

### Checklist

- [ ] He leído las normas de contribución del proyecto.
- [ ] He verificado que este problema/tarea no ha sido previamente reportado/a.
- [ ] He verificado que este problema/tarea no está actualmente en proceso por otro contribuidor.
- [ ] Estoy asignándome a mí mismo/a este issue o se lo estoy asignando al miembro del equipo adecuado.
- [ ] I have read the project's contribution guidelines.
- [ ] I have checked that this issue/task has not been previously reported.
- [ ] I have verified that this issue/task is not currently being addressed by another contributor.
- [ ] I am assigning this issue to myself or to the appropriate team member.
3 changes: 3 additions & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ habi.json
habi.jsonl
data/raw/*.json
logs/*
bogota_apartments/spiders/gojom.py
.vscode/
data/interim/*.csv

### JupyterNotebooks ###
# gitignore template for Jupyter Notebooks
Expand Down
167 changes: 28 additions & 139 deletions CODE_OF_CONDUCT.md
View file
Original file line number Diff line number Diff line change
@@ -1,167 +1,56 @@
# Contributor Covenant Code of Conduct

The scraper used in the "**Bogota Apartments**" repository is a powerful tool
that collects data from the real estate market of apartments in Bogota.
To ensure a responsible and ethical use of this tool, it is important to follow
this code of conduct:
# Code of Conduct for Ethical Use of the Scraper

- **Legal compliance:** use the scraper in a legal manner and in compliance
with all applicable laws and regulations. Do not engage in illegal activities,
such as unauthorized access to websites, extraction of copyrighted information
or any other activity that violates the law.
The scraper used in the "**Bogotá Apartments**" repository is a powerful tool that collects data from the real estate market of apartments in Bogotá. To ensure a responsible and ethical use of this tool, it is important to follow this code of conduct:

- **Respect terms of service:** Be sure to respect the terms of service of the websites
from which data is extracted. If a website has specific restrictions or policies
regarding data extraction, respect them and adjust the scraper accordingly.
## Legal and Ethical Responsibilities

- **Speed and frequency limitations:** Configure the scraper so as not to overload
or negatively affect the performance of the target websites. Adjust the speed
and frequency of extraction to avoid jeopardizing the availability or proper
functioning of the websites.
- **Legal Compliance:** Use the scraper in a legal manner and in compliance with all applicable laws and regulations. Avoid illegal activities such as unauthorized access to websites, extraction of copyrighted information, or any other activity that violates the law.

- **Privacy and Confidentiality:** Respect the privacy and confidentiality of collected data.
Do not share personal or confidential information obtained through the scraper without proper
consent and necessary safeguards.
- **Respect Terms of Service:** Be sure to respect the terms of service of the websites from which data is extracted. If a website has specific restrictions or policies regarding data extraction, respect them and adjust the scraper accordingly.

- **Transparency in data collection:** If you share or publish data collected with the scraper,
be sure to provide proper attribution and references to the original sources. Do not present
the data as your own if it was obtained through the tool.
- **Speed and Frequency Limitations:** Configure the scraper so as not to overload or negatively affect the performance of the target websites. Adjust the speed and frequency of extraction to avoid jeopardizing the availability or proper functioning of the websites.

- **Maintain and improve the scraper:** If you are responsible for developing and maintaining the scraper,
make sure to perform regular updates and fix any issues or bugs detected. Encourage code transparency
and facilitate collaboration with other community members to improve the tool.
## Privacy and Confidentiality

- **Report issues and suggestions:** If you encounter problems or have suggestions for improving the scraper,
report them constructively in the repository. Participate in open and collaborative discussions to help maintain
and improve the quality and efficiency of the scraper.
- **Respect Privacy:** Respect the privacy and confidentiality of collected data. Do not share personal or confidential information obtained through the scraper without proper consent and necessary safeguards.

Remember that the use of the scraper must be responsible and ethical. By following this code of conduct,
you contribute to ensure a proper use of the tool and to maintain an environment of respect
and compliance with the rules in the repository "**Bogotá Apartments**".
- **Transparency in Data Collection:** If you share or publish data collected with the scraper, be sure to provide proper attribution and references to the original sources. Do not present the data as your own if it was obtained through the tool.

## Our Pledge
## Maintenance and Improvement of the Scraper

We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity
and orientation.
- **Maintain and Improve:** If you are responsible for developing and maintaining the scraper, make sure to perform regular updates and fix any issues or bugs detected. Encourage code transparency and facilitate collaboration with other community members to improve the tool.

We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.
## Reporting Issues and Suggestions

## Our Standards
- **Constructive Communication:** If you encounter problems or have suggestions for improving the scraper, report them constructively in the repository. Participate in open and collaborative discussions to help maintain and improve the quality and efficiency of the scraper.

Examples of behavior that contributes to a positive environment for our
community include:
Remember that the use of the scraper must be responsible and ethical. By following this code of conduct, you contribute to ensuring a proper use of the tool and to maintain an environment of respect and compliance with the rules in the "**Bogotá Apartments**" repository.

* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the
overall community

Examples of unacceptable behavior include:
# Código de Conducta para el Uso Ético del Scraper

* The use of sexualized language or imagery, and sexual attention or
advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email
address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
El scraper empleado en el repositorio "**Bogotá Apartments**" es una herramienta poderosa que recopila datos del mercado inmobiliario de apartamentos en Bogotá. Para garantizar un uso responsable y ético de esta herramienta, es crucial seguir este código de conducta:

## Enforcement Responsibilities
## Responsabilidades Legales y Éticas

Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.
- **Cumplimiento Legal:** Utiliza el scraper de manera legal y conforme a todas las leyes y regulaciones aplicables. Evita actividades ilegales como acceso no autorizado a sitios web, extracción de información protegida por derechos de autor u otras acciones que violen la ley.

Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.
- **Respeto a los Términos de Servicio:** Asegúrate de respetar los términos de servicio de los sitios web de los que se extraen datos. Si un sitio tiene restricciones específicas o políticas sobre la extracción de datos, respétalas y ajusta el scraper en consecuencia.

## Scope
- **Limitaciones de Velocidad y Frecuencia:** Configura el scraper para no sobrecargar ni afectar negativamente el rendimiento de los sitios web objetivo. Ajusta la velocidad y frecuencia de extracción para no comprometer la disponibilidad o el funcionamiento adecuado de los sitios.

This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.
## Privacidad y Confidencialidad

## Enforcement
- **Respeto a la Privacidad:** Respeta la privacidad y confidencialidad de los datos recopilados. No compartas información personal o confidencial obtenida a través del scraper sin el consentimiento adecuado y las salvaguardias necesarias.

Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
[email protected].
All complaints will be reviewed and investigated promptly and fairly.
- **Transparencia en la Recopilación de Datos:** Si compartes o publicas datos recopilados con el scraper, asegúrate de proporcionar la atribución adecuada y referencias a las fuentes originales. No presentes los datos como propios si fueron obtenidos a través de la herramienta.

All community leaders are obligated to respect the privacy and security of the
reporter of any incident.
## Mantenimiento y Mejora del Scraper

## Enforcement Guidelines
- **Actualización y Mantenimiento:** Si eres responsable del desarrollo y mantenimiento del scraper, realiza actualizaciones periódicas y soluciona cualquier problema o error detectado. Fomenta la transparencia del código y facilita la colaboración con otros miembros de la comunidad para mejorar la herramienta.

Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:
## Reporte de Problemas y Sugerencias

### 1. Correction
- **Comunicación Constructiva:** Si encuentras problemas o tienes sugerencias para mejorar el scraper, repórtalos de manera constructiva en el repositorio. Participa en discusiones abiertas y colaborativas para ayudar a mantener y mejorar la calidad y eficiencia del scraper.

**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.

**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.

### 2. Warning

**Community Impact**: A violation through a single incident or series
of actions.

**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or
permanent ban.

### 3. Temporary Ban

**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.

**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.

### 4. Permanent Ban

**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.

**Consequence**: A permanent ban from any sort of public interaction within
the community.

## Attribution

This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.0, available at
https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.

Community Impact Guidelines were inspired by [Mozilla's code of conduct
enforcement ladder](https://github.com/mozilla/diversity).

[homepage]: https://www.contributor-covenant.org

For answers to common questions about this code of conduct, see the FAQ at
https://www.contributor-covenant.org/faq. Translations are available at
https://www.contributor-covenant.org/translations.
Recuerda que el uso del scraper debe ser responsable y ético. Al seguir este código de conducta, contribuyes a garantizar un uso adecuado de la herramienta y a mantener un entorno de respeto y cumplimiento con las reglas en el repositorio "**Bogotá Apartments**".
Loading

0 comments on commit 32cbd90

Please sign in to comment.