Reuse Record by Jannik Oslender

[valiantone]

What's the problem you are trying to solve?

Baukreisel is an interdisciplinary collective and non profit working towards enabling reuse in the construction industry at scale. Material category after material category we are developing rationalized reuse processes for cataloging/recording, extracting, transporting, refurbishing and reinserting building material and elements.Due to their high carbon impact and the potential to be reused at scale our current research projects are focused on concrete primary structures and windows elements. The research initiative is called reuse.matters.In the reuse process the recording and surveying of the elements is a very time consuming and tedious task.The current methods (Excel or Airtable) do not offer any level of automation and require you to manually input information for similar elements again and again. Besides that, the databases cannot store the different sub elements, that e.g. a window consists of, in a machine readable and understandable way.This knowledge about the interconnection of elements and subelements is especially important when refurbishing or transforming. For example when dealing with windows the glass and the frame might be reused separately and in a different way. Thus information about the subelement is important and should be derived as implicit knowledge.In short the project should solve 3 problems:-

1. Create a way to easily record and survey building elements and material in soon to be demolished buildings
2. Make that process a lot faster (e.g. through a convenient interface, smart templates and live recommendations/autofills as well as computer vision and object recognition.
3. Save the information and relations in a semantic and machine readable way so that they are available in the reuse process

What's your idea for a solution to the given "problem"?

A mobile app to record and survey building elements on site and a graph database to store the information so that later on explicit and implicit knowledge can be retrieved.In order to being a feasible project, that can be developed in the scope and duration of the project phase the app shall be focus on the surveying of window elements.To speed up the recording process there are three key elements:- An easy to use interface that quickly guides you through the recording process- Smart autofills and suggestions that reduce the need to input the same information several times. An example could be that the building location and the current storey you are in is prefilled with information from the last entry. Similarly element types/categories could be suggested while typing etc.- A catalogue of possible shapes and window configurations (amount of windows sashes, fanlight, door etc.) and object recognition that suggest, based upon the catalogue, the type of configuration to the user after taking a photo. Retrieving information about the frame colour or material would be great but maybe more difficult to retrieve.(- A way to quickly get rough measurements using the phones camera and AR features would be great but very likely beyond the scope of the project phase so just as a note)Regarding storing the data, the building industry has developed several semantic ontologies to structure the data that could be used as the foundation for the layout of the graph database. In contrast to relational databases, graph databases allow you to very easily store relationships (I know, somewhat unintuitive). This is espacially important when working with buildings and materials. There are geometrical relationships (room is part of a storey is part of a building) and type relationships (oak is a type of hardwood is a type of wood) and subelement relationships (glass and frame are a subelement of a casement is a sub element of a window is a sub element of the facade).This can be modeled in a graph like neo4j and not only explicit but also implicit knowledge can be retrieved (amount of glass elements inside a storey) (all wooden window frames in a building etc.).Even though graph databases offer a lot of advantages in this scenario their implementation might be beyond the scope of the project phase and as an MVP the app could also work with a relational database.

Which tracks do you think could be involved?

DL/DS
WD
UX

How do you imagine Deep Learning and/or Data Science could contribute to this project?

Through setting up and training a neural network that allows the app to recognize the shape and configuration of the photographed windows elements.

How do you imagine User Experience Design could contribute to this project?

Though setting up an easy to use interface that lets the user quickly cycle through the recording steps.

How do you imagine Web Development could contribute to this project?

Through setting up a front end that autofills and suggests inputs to make the recording and surveying process faster.Through setting up a backend that allows the information and relationships to be stored in a graph database.

What are the most important challenges & bottlenecks of your project?

• Understanding the recording and surveying process so that it can be meaningfully improved through the app- creating the logic behind the autofill algorithm and connecting that to the data layout and interface- setting up the structure and semantics of the database and connecting it to the frontend- getting a large variety of photos of different window frames and label them according to the catalogue of possibilities- achieving a high enough success rate for the neural net to recognize size and configuration

What are the most important success factors of your project?

Teamwork:Only if different people from the different tracks work well together the project can realize its full potentialPractical insight:Baukreisel is going to supply us as good as possible with all the real world information about the recording process.

Share your resources here:

Links:

• About Baukreisel
• Baukreisel Insta
• about reuse.matters
• Screenshot of current inventory tracking that I have build using Airtable

[valiantone]

Ok, so the scope of this project may be too large to achieve in 10 weeks, let's make a first start towards a MVP that should be usable and solve the core need, this would require:

UX

The task would be straight-forward to implement however the app seems to be for expert users in a specific domain. Can you guarantee the UX techies access to test users (3-5 users) who would be willing to test and give feedback? This is mandatory for the UX process.

WD

Also a straight-forward task, once you lockdown a few features for development the FE and BE techies can divide the required tasks between themselves. Maybe already take some time to think of features that would be required (eg search over graph database, dashboard view, history view, photo upload etc) for the app. Final decisions may differ, but it's good to list some prospective ideas to start.

Please also remember only a webapp would be within the scope of project phase, native application development has not bee covered in our curriculum.

DS/DL

Please differentiate between tasks that would be performed by DL techies (building image classification to detect and identify window elements) vs DS techies (auto-suggestion and completion for analytical and ml-based inference of parameters in templates). Here again, final features are subject to change but would be good to have a clear initial proposal on this from project requestor.

Simply, respond to the comments @Jannik0s and we can close this conversation and move your project to Accepted so you are all clear for team formation and project phase! 🚀

[Jannik0s]

I totally agree! The topic is super vast but concentrating on windows as the element to record and prioritizing features might make it feasible.

The web app quite likely consist of 3 parts

1 Input:
This is where the recording and surveying happens.
I think it is the most important part since this is currently the biggest pain point.

2 Catalogue:
Here you can see the contents of the Graph Database.
Important to have but maybe a bit less challenging to develop since there are a lot of examples for it

3 Dashboard:
Get analytics about the stock of materials/building elements that have been recorded.
This would add a lot of value but would not be necessary for the app to work.

UX

The collective has a core team of 6 including me and everybody is eager to support us with knowledge and testing.
However, since there are buildings - and windows for that matter - all around us, everybody else can give it a test at home too.

WD

1 Input:
Necessary:

• Form to input the different properties of the window element
• Photo Upload

Important:

• Preview of the window element configuration
• Deriving measurements for parts (sashes, frame, window pane) from the measurements and properties (amount of sashes, fanlights, doors etc.) of the bigger window element (might be a DS task)

2 Catalogue:
Necessary:

• List view of all the elements recorded
• Search field to find / filter items by material type / amount of elements / frame colour / size etc

Important:

• Sorting the items in relation to the different properties
• really harness the power of graph databases to find and list e.g. the glass elements that are part of the bigger window element independently (Thus overcoming the hurdle that you now need to catalogue every single part of an element to find it later on)

Great to have:

• Gallery view with photos

3 Dashboard:
Necessary:

• Analytics of material in stock in the form of pie chart / bar graph etc. (See Link)

Important:

• use implicit knowledge of graph database to get the analytics from subelements (xx m² of window panes, xx m³ wooden frames, xx m of frame profiles etc.) (might be a DS task)

DS

1 Input:
Necessary:

• structure the data that is delivered trough the form
• derive properties / measurements from the inserted data (window is 1x3 m and horizontally divided into 3 sashes -> each sash is 1x1 -> standard frame size is 0,07 cm -> 3 window panes 0,93 x 093 m ...)

Important:

• Pre-fill the building and the storey from the previously recorded element
• Auto suggest properties depending on the items that have been catalogued previously (like Google search bar)

2 Catalogue:
Necessary:

• supply the structured data so that it can be shown in the gallery view
• respond to the search queries from the search bar

Important:

• restructure the data for the different sorting types of the list view
• show the implicit knowledge in the database
  (when searching for wood show window frames that consist of oak, teak, beech etc.)
  (when searching for window panes show the properties of the window panes that are part of a window element)
  (when searching for material in a city show the stocks in different buildings in the city)

3 Dashboard:
Necessary:

• Calculate the data for the charts and graphs

Important:

• use implicit knowledge of graph database to get the analytics from subelements (xx m² of window panes, xx m³ wooden frames, xx m of frame profiles etc.)

DL

1 Input:
Necessary:

• Build image classification to detect and identify window elements
• Provide that data so that it can be used as a pre-fill in the Input form and is thus adjustable/correctable by the user
• Dataset of labeled photos could be created through fotos of everybody's own windows (Baukreisel members & Techies)
• Properties to recognize (window configuration [amount / arrangement of sashes, doors, fanlights], colour of the frame)

[valiantone]

All right @Jannik0s, quite some detailed answers, looks like we've thought of most of the requirements and have some starting points for all discussions. Please feel free to start looking for team mates! I would suggest a standard team setup for your need:

Team Formation Requirements

• 1-2 DL
• 2 DS
• 2 UX
• 1BE (WD)
• 1FE (WD)

[Jannik0s]

This is great news!

@martajasinska already expressed her interest and is joining as UX Techie!
So good to have you in the Team, Marta  🥳🚀

I am going to take part as Backend WD so still positions of every track vacant.

Really looking forward to you guys reaching out!

[Jannik0s]

On this Miro we have started collecting relevant information for the project:

https://miro.com/app/board/uXjVN2Hi_FY=/?share_link_id=121195818869

The Password is:
TechLabs

[Jannik0s]

Happy to announce that @pr4t1ma joined the team as Front End WD.
Looking forward to making this project a reality with you!

Soma, Christoph and Arpad expressed their interest to become part of the project as DL Techies.
It is so great that you would like to push it from a Deep Learning perspective!

Since there are currently only 1-2 seats assigned for DL maybe the Track Leads @valiantone can give some guidance on how to proceed.

[Jannik0s]

Reuse Record 🌀 Team

• 1-2? DL @somahargitai & @Risto
• 2 DS
• 2 UX @martajasinska & @mahanfa
• 1 WD (BE) @Jannik0s
• 1 WD (FE) @pr4t1ma

[Jannik0s]

DL

During the first discussions with DL Techie Christoph, some questions about the dataset to train the image recognition model have come up.
Unfortunately there is no dataset that we are aware of that labels the images in the way we need it.

Since there is still some time before the project starts and since through Baukreisel we have a couple of people at hand, is it possible to build and label that dataset on our own?

I think it would be quite doable to have a couple of hundred labeled pictures ready before the project starts.
Especially considering it is possible to include photos of the same window from different angles.

[martajasinska]

I would add the opening direction DIN left/ DIN right/ bottom hung window (Kippfenster)/ fix-glazed window (Festverglasung) to the window parameters - this should be easily determinable based on the position of the handle

[Jannik0s]

That would be nice to have indeed but classifying is getting complicated when the windows consist of several sub elements like the first one.

Would then be something like
(1l + 1r + 1r) x (1l + 1r +1l)

Probably we have to focus on simpler windows first anyways.
Would still be nice to know how to set this up so that it can be extended.

[mahanfa]

Hello, nice project! I've studied Architecture as well. I would like to join as a UX designer if possible, thanks!

[Jannik0s]

That's great!
Welcome to the team!

[mahanfa]

Hey guys, I'm very sorry, just in the last minute I understood that we can only be in one project. I had my own project and I thought I have to also take part in someone else's project. So I have to leave this one. :( Just for the record if it is for any good, my very good friend Mana is looking for projects so I will mention her here. @Mananosrati

[Mananosrati]

Hi there,
it's an amazing project and idea. if it is possible I would like to join you as a UX member.