Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/cifar 10 #480

Merged
merged 2 commits into from
Jul 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions episodes/4-advanced-layer-types.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -47,15 +47,15 @@
### A Note About Data Provenance
In an earlier version, this part of the lesson used a different example dataset.
During [peer review](https://github.com/carpentries-lab/reviews/issues/25#issuecomment-1953271802), the decision was made to replace that dataset due to the way it had been compiled using images "scraped" from the internet without permission from or credit to the original creators of those images. Unfortunately, uncredited use of images is a common problem among datasets used to benchmark models for image classification.
The Dollar Street Dataset was chosen for use in the lesson as it contains only images [created by the Gapminder project](https://www.gapminder.org/dollar-street/about?) expressly for the purposes of use in the dataset.
The original Dollar Street Dataset is very large -- more than 100 GB -- with the potential to grow even bigger, so we created a subset for use in this lesson.
The Dollar Street dataset was chosen for use in the lesson as it contains only images [created by the Gapminder project](https://www.gapminder.org/dollar-street/about?) expressly for the purposes of use in the dataset.
The original Dollar Street dataset is very large -- more than 100 GB -- with the potential to grow even bigger, so we created a subset for use in this lesson.

::::::::::::::::::::::::::::::::::::::::::::::::::


## Dollar Street 10 dataset

The dollar street 10 dataset consists of images of 10 different classes, this is the mapping of the categories:
The Dollar Street 10 dataset consists of images of 10 different classes, this is the mapping of the categories:

| Category | label |
|-----------------|-------|
Expand All @@ -80,7 +80,7 @@

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

![Sample images from the dollar street 10 data-set. Each image is labelled with a category, for example: 'street sign' or 'soap dispenser'](fig/04_dollar_street_10.png){alt="A 5 by 5 grid of 25 sample images from the dollar street 10 data-set. Each image is labelled with a category, for example: 'street sign' or 'soap dispenser'."}
![Sample images from the Dollar Street 10 dataset. Each image is labelled with a category, for example: 'street sign' or 'soap dispenser'](fig/04_dollar_street_10.png){alt="A 5 by 5 grid of 25 sample images from the dollar street 10 data-set. Each image is labelled with a category, for example: 'street sign' or 'soap dispenser'."}

## 2. Identify inputs and outputs

Expand All @@ -101,9 +101,9 @@

::: challenge

### Number of features dollar-street-10
### Number of features in Dollar Street 10

How many features does one image in the dollar-street-10 dataset have?
How many features does one image in the Dollar Street 10 dataset have?

- A. 64
- B. 4096
Expand Down Expand Up @@ -263,7 +263,7 @@

::: challenge
## Number of model parameters
Suppose we apply a convolutional layer with 100 kernels of size 3 * 3 * 3 (the last dimension applies to the rgb channels) to our images of 32 * 32 * 3 pixels. How many parameters do we have? Assume, for simplicity, that the kernels do not use bias terms. Compare this to the answer of the earlier exercise, ["Number of Parameters"](#parameters-exercise-1).

Check warning on line 266 in episodes/4-advanced-layer-types.Rmd

View workflow job for this annotation

GitHub Actions / Build markdown source files if valid

[missing anchor]: ["Number of Parameters"](#parameters-exercise-1)

:::: solution
## Solution
Expand Down
6 changes: 3 additions & 3 deletions episodes/5-transfer-learning.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ In this episode we will learn how use Keras to adapt a state-of-the-art pre-trai
## 1. Formulate / Outline the problem and 2. Identify inputs and outputs


Just like in the previous episode, we use the dollar street 10 dataset.
Just like in the previous episode, we use the Dollar Street 10 dataset.
The goal is to predict one out of 10 classes for a given image.

We load the data in the same way as the previous episode:
Expand Down Expand Up @@ -80,7 +80,7 @@ base_model = keras.applications.DenseNet121(include_top=False,
```
By setting `include_top` to `False` we exclude the fully connected layer at the
top of the network. This layer was used to predict the Imagenet classes,
but will be of no use for our dollarstreet dataset.
but will be of no use for our Dollar Street dataset.

We add `pooling='max'` so that max pooling is applied to the output of the DenseNet121 network.

Expand All @@ -94,7 +94,7 @@ Instead of fine-tuning all the weights of the DenseNet121 network using our data
we choose to freeze all these weights and only train a so-called 'head network'
that sits on top of the pre-trained network. You can see the DenseNet121 network
as extracting a meaningful feature representation from our image. The head network
will then be trained to decide on which of the 10 dollarstreet classes the image belongs.
will then be trained to decide on which of the 10 Dollar Street dataset classes the image belongs.

We will turn of the `trainable` property of the base model:
```python
Expand Down
3 changes: 2 additions & 1 deletion episodes/6-outlook.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ It is part of the codebase for [this paper](https://doi.org/10.1186/s13321-021-0
In short, the deep learning problem is that of finding out how similar two molecules are in terms of their molecular properties,
based on their mass spectrum.
You can compare this to comparing two pictures of animals, and predicting how similar they are.

A siamese neural network is used to solve the problem.
In a siamese neural network you have two input vectors, let's say two images of animals or two mass spectra.
They pass through a base network. Instead of outputting a class or number with one or a few output neurons, the output layer
Expand Down Expand Up @@ -94,7 +95,7 @@ In recent years the quality of LLMs language understanding and generation has in
It is becoming more and more feasible to unleash this power in scientific research. For example, the authors of [Zheng et al. (2023)](https://doi.org/10.1021/jacs.3c05819) guided ChatGPT in the automation of extracting chemical information from a large amount of research articles. The authors did not implement a deep learning model themselves, but instead they designed the right input for ChatGPT (called a 'prompt') that would produce optimal outputs. This is called prompt engineering. A highly simplified example of such a prompt would be: "Given compounds X and Y and context Z, what are the chemical details of the reaction?"

Developments in LLM research are moving fast, at the end of 2023 the newest ChatGPT version [could take images and sound as input](https://openai.com/blog/chatgpt-can-now-see-hear-and-speak).
In theory, this means that you can solve the Cifar-10 image classification problem from the previous episode by prompt engineering, with prompts similar to "Which out of these categories: [LIST OF CATEGORIES] is depicted in the image".
In theory, this means that you can solve the Dollar Street image classification problem from the previous episode by prompt engineering, with prompts similar to "Which out of these categories: [LIST OF CATEGORIES] is depicted in the image".

**Discuss the following statement with your neighbors:**

Expand Down
Loading