Civil Work Bidding Price Prediction in San Francisco. AI-Powered.🌉🏗️ 💸

• App

In case of 404, click on video capture

Table of contents

• Table of contents
• Background
• Project
• Key documents
  • Top-directory layout
• Technologies
• Datasets
• Getting started
  • Ensure that pip module is installed by running :
  • How to install pipenv ?
  • Clone the repository :
  • Create your virtual environment :
  • Install all required librairies :
• Install the project
• License
• Authors
• Contact
• Made with ❤️ in Paris

Background

San Francisco is a hyper-popular city with homeless community (20% of population), natural disaster risks and astronomical housing prices. Affordable housing in San Francisco has not been an option. Meanwhile, new affordable construction projects are high in need. Many investors consider construction projects to invest in SF, which can provide high return rate. For construction projects, engineers struggle to predict the construction project cost as reasonable as possible to win biddings. 🏗️ 💸

Project

This project is an AI-powered app 🧠🤖 to estimate cost of construction projects in San Francisco.

• Presentation of project

Some of the efforts include data cleaning, feature engineering, setting up machine learning models, predictive error calculation, parameter tuning, creating a dashboard and deploying an online app.

Machine learning models were trained with the historical data coming from building permits of San Francisco available since early 1980s (thanks to datasf.org). A set of parameters and machine learning models were tested (including Linear,Lasso model, E-Net, KRidge, GBoosting, XGBoost, LGBoost and Random Forest). Random Forest Model🌲🌳🌲🌳 was judged to use as a final model.

Structural work cost

✅ include cost of foundation, columns, beams, slabs, floors, roof and workmanship cost

❌ does not include land price, finishing work, electricity & plumbing and commercial costs

Key documents

1 - Notebook on exploratory data analysis

2 - Script on data cleaning

3 - Notebook on machine learning predictive models

4 - Logs on scores of machine learning models and experiments

Top-directory layout

.
├── Docker                  # App deployment. Scripts with streamlit, final ML models and Docker container settings.
├── NoteBooks               # Jupyter notebooks on EDA, feature engineering and ML models
├── ShapeOut                # Documents on building footprints in SF
├── Tracking                # Logs of ML model scores and experiments
├── src                     # Scripts on functions, database cleaning, building ground surface area and ML model experiments 
├── LICENSE
├── README.md 
├── README_FR.md
└── requirements.txt

Technologies

Project is created with:

• Python 3.8
• Jupyter Notebook 6.4.12
• Python libraries (see /requirements.txt)
• Streamlit 1.12.0
• Docker 20.10.18
• VSCode 1.71.2

Datasets

1 - San Francisco Permit Data

A building permit is an official approval document issued by a government agency that allows a construction or a renovation project on a property. More information can be found on this website. Each city or county has its own building office to perform multiple functions such as issuing permits, inspecting buildings for safety measures, changing rules to meet the needs of a growing population, etc. For the City of San Francisco, building permits are handled by SF DBI. The dataset includes details on application/permit ID, job location, the current status of the applications and some other details. Data is uploaded weekly by DBI.

2 - Building Footprints in San Francisco

Getting started

To run this project you need :

• Python 3.10
• pip
• pipenv (optional)

Ensure that pip module is installed by running :

```sh
python -m ensurepip --default-pip

How to install pipenv ?

pip install pipenv

Clone the repository :

git clone https://github.com/LHB-Group/Civil-Work-Bidding-And-Investment-Helper.git

Create your virtual environment :

Go to your repository folder and run the following command.

python -m venv my_venv
# the my_venv folder should appear

Once created you need to activate your venv

Windows :

# Go to your repository folder and type :
./my_venv/Scripts/activate
# You should have something like :
(my_venv) C:\Users\...\Civil-Work-Bidding-And-Investment-Helper>

Linux / Mac OS :

# Go to your repository folder and type :
source my_venv/bin/activate
# You should have something like :
(my_venv) C:\Users\...\Civil-Work-Bidding-And-Investment-Helper>

Install all required librairies :

pip install -r requirements.txt

Install the project

The install.py script permit you to dowload and create all folders and files for the project.

Build the complete dataset can take several hours because of feature engineering but you can download it here and move it to [Datasets] Folder

# go to the src folder and run :
python ./install.py

If you download the File you need to rename it :

Exemple : 20220823_Building_Permits.csv (CreationDate+FileName)

You need to rename it to Building_Permits.csv

License

Distributed under the MIT License. See LICENSE.txt for more information.

Authors

Baptiste Cournault :

GitHub

Hicham Mrani :

GitHub

Levent ISBILIROGLU :

GitHub

Contact

Please see contact details on presentation file above.

---

Made with ❤️ in Paris