Skip to content

Commit

Permalink
Merge pull request #14 from Emerald-Z/main
Browse files Browse the repository at this point in the history 
XGboost Changes for ocean-prefilter
  • Loading branch information
@Emerald-Z
Emerald-Z authored Jul 23, 2024
2 parents efe23e3 + 83f26a7 commit 1c90b5c
Show file tree
Hide file tree
Showing 14 changed files with 986 additions and 304 deletions.
13 changes: 10 additions & 3 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ This module implements the [`rdk:service:vision` API](https://docs.viam.com/ml/v

When you configure a machine with this module, the module:
- Locates the horizon, crops the image to only include the water, and then divides the water into patches.
- Turns the resulting patches into histograms, which it interprets as probability distrubutions for their color/intensity values.
- Conducts a Kolmogorov-Smirnov test on the histograms from before and after images and determines if the scenes are consistent or if a new element has appeared.
- Performs feature extraction on the resulting patches, average pooling the the patches with a window size of (10, 2) and taking the mean of the R, G, B channels for each resulting sub-patch
- Classifies the frame using XGBoost - this will trigger if any patch in the given image is found "interesting"

Strong motion of the waves or bobbing up-and-down of the boat can trigger the pre-filter.
This vision service only returns one label classification called `TRIGGER` with a confidence of `1.0`.
Expand Down Expand Up @@ -56,7 +56,14 @@ Copy and paste the following attribute template into your vision service's attri

| Name | Type | Inclusion | Description | Value |
|-------|-------|-----------|-------------| ------|
| `camera_name` | string | **Required** | Links the pre-filter to a specific camera and continuously monitors the camera stream for changes or triggers in the background. | The name of your camera component. |
| `camera_name` | string | Optional | Links the pre-filter to a specific camera and continuously monitors the camera stream for changes or triggers in the background. | The name of your camera component. If the camera name is not provided, you can input your own image from the VIAM API |
| `threshold` | int | Optional | Determines the sensitivity of the pre-filter trigger. This enables the pre-filter to detect significant motion such as boat or wave movements, and identifies objects like other boats, buoys, or any deviations from typical water patterns. | 0 to 1<br/> Default: `0.25` |
| `max_frequency_hz`| int | Optional | Determines the frequency that the vision service monitors the background camera stream for changes. If your scene changes very slowly set this below 1. | 1 to 10<br/> Default: `10` |
| `excluded_region` | object | Optional | Specifies areas within the cameras view to ignore. This is useful for excluding static parts of the camera stream, like parts of the boat. | A list of coordinates in frame. |

### Example
The test module example gives an example of how to run/use the service
provide your test directory to the module in the form of a command line argument
```
run main.go <your directory here>
```
53 changes: 53 additions & 0 deletions example/main.go
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
package main

import (
_ "embed"
"fmt"
"image"
"os"
"path/filepath"

xgb "github.com/Elvenson/xgboost-go"
"github.com/Elvenson/xgboost-go/activation"
"github.com/viamrobotics/ocean-prefilter/oceanprefilter"
)

//go:embed xg_boost_dump.json
var modelbytes []byte

func main() {
dir := os.Args[1]
fmt.Println(dir)
files, _ := os.ReadDir(dir)

rc := oceanprefilter.RunConfig{}
ensemble, err := xgb.LoadXGBoostFromJSONBytes(modelbytes,
"", 2, 8, &activation.Softmax{})

if err != nil {
fmt.Println(err)
}

rc.Model = ensemble
rc.Threshold = 0.25
rect := image.Rectangle{
Min: image.Point{X: 250, Y: 350},
Max: image.Point{X: 580, Y: 480},
}

rc.ExcludedZone = &rect
for _, file := range files {
fp := filepath.Join(dir, file.Name())
f, _ := os.Open(fp)

defer f.Close()
img, _, _ := image.Decode(f)

res, err := oceanprefilter.MakeInference(img, rc)
if err != nil {
fmt.Println("error making inference: ", err)
} else {
fmt.Printf("output is %t: ", res)
}
}
}
20 changes: 20 additions & 0 deletions example/test.libsvm
View file

Large diffs are not rendered by default.

142 changes: 142 additions & 0 deletions example/xg_boost_dump.json
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
[
{ "nodeid": 0, "depth": 0, "split": "f327", "split_condition": 110.783333, "yes": 1, "no": 2, "missing": 2 , "children": [
{ "nodeid": 1, "depth": 1, "split": "f14", "split_condition": 199.883331, "yes": 3, "no": 4, "missing": 4 , "children": [
{ "nodeid": 3, "depth": 2, "split": "f436", "split_condition": 108.216667, "yes": 7, "no": 8, "missing": 8 , "children": [
{ "nodeid": 7, "leaf": 0.96825397 },
{ "nodeid": 8, "leaf": -0 }
]},
{ "nodeid": 4, "leaf": -0 }
]},
{ "nodeid": 2, "depth": 1, "split": "f628", "split_condition": 107.199997, "yes": 5, "no": 6, "missing": 6 , "children": [
{ "nodeid": 5, "leaf": -0.800000012 },
{ "nodeid": 6, "leaf": 0.200000003 }
]}
]},
{ "nodeid": 0, "depth": 0, "split": "f327", "split_condition": 110.783333, "yes": 1, "no": 2, "missing": 2 , "children": [
{ "nodeid": 1, "depth": 1, "split": "f14", "split_condition": 199.883331, "yes": 3, "no": 4, "missing": 4 , "children": [
{ "nodeid": 3, "depth": 2, "split": "f436", "split_condition": 108.216667, "yes": 7, "no": 8, "missing": 8 , "children": [
{ "nodeid": 7, "leaf": -0.96825397 },
{ "nodeid": 8, "leaf": -0 }
]},
{ "nodeid": 4, "leaf": -0 }
]},
{ "nodeid": 2, "depth": 1, "split": "f628", "split_condition": 107.199997, "yes": 5, "no": 6, "missing": 6 , "children": [
{ "nodeid": 5, "leaf": 0.800000012 },
{ "nodeid": 6, "leaf": -0.200000003 }
]}
]},
{ "nodeid": 0, "depth": 0, "split": "f532", "split_condition": 140.75, "yes": 1, "no": 2, "missing": 2 , "children": [
{ "nodeid": 1, "depth": 1, "split": "f28", "split_condition": 204.416672, "yes": 3, "no": 4, "missing": 4 , "children": [
{ "nodeid": 3, "depth": 2, "split": "f401", "split_condition": 111.400002, "yes": 5, "no": 6, "missing": 6 , "children": [
{ "nodeid": 5, "leaf": 0.570969284 },
{ "nodeid": 6, "leaf": -0.183552772 }
]},
{ "nodeid": 4, "leaf": -0.117992692 }
]},
{ "nodeid": 2, "leaf": -0.571418047 }
]},
{ "nodeid": 0, "depth": 0, "split": "f532", "split_condition": 140.75, "yes": 1, "no": 2, "missing": 2 , "children": [
{ "nodeid": 1, "depth": 1, "split": "f28", "split_condition": 204.416672, "yes": 3, "no": 4, "missing": 4 , "children": [
{ "nodeid": 3, "depth": 2, "split": "f401", "split_condition": 111.400002, "yes": 5, "no": 6, "missing": 6 , "children": [
{ "nodeid": 5, "leaf": -0.570969284 },
{ "nodeid": 6, "leaf": 0.183552772 }
]},
{ "nodeid": 4, "leaf": 0.117992707 }
]},
{ "nodeid": 2, "leaf": 0.571418047 }
]},
{ "nodeid": 0, "depth": 0, "split": "f532", "split_condition": 140.75, "yes": 1, "no": 2, "missing": 2 , "children": [
{ "nodeid": 1, "depth": 1, "split": "f28", "split_condition": 204.416672, "yes": 3, "no": 4, "missing": 4 , "children": [
{ "nodeid": 3, "depth": 2, "split": "f410", "split_condition": 110.650002, "yes": 5, "no": 6, "missing": 6 , "children": [
{ "nodeid": 5, "leaf": 0.466001004 },
{ "nodeid": 6, "leaf": -0.0753230453 }
]},
{ "nodeid": 4, "leaf": -0.0537564792 }
]},
{ "nodeid": 2, "leaf": -0.356984079 }
]},
{ "nodeid": 0, "depth": 0, "split": "f532", "split_condition": 140.75, "yes": 1, "no": 2, "missing": 2 , "children": [
{ "nodeid": 1, "depth": 1, "split": "f28", "split_condition": 204.416672, "yes": 3, "no": 4, "missing": 4 , "children": [
{ "nodeid": 3, "depth": 2, "split": "f410", "split_condition": 110.650002, "yes": 5, "no": 6, "missing": 6 , "children": [
{ "nodeid": 5, "leaf": -0.466000974 },
{ "nodeid": 6, "leaf": 0.0753231421 }
]},
{ "nodeid": 4, "leaf": 0.0537564307 }
]},
{ "nodeid": 2, "leaf": 0.356984049 }
]},
{ "nodeid": 0, "depth": 0, "split": "f284", "split_condition": 101.400002, "yes": 1, "no": 2, "missing": 2 , "children": [
{ "nodeid": 1, "leaf": 0.347005337 },
{ "nodeid": 2, "depth": 1, "split": "f300", "split_condition": 95.5, "yes": 3, "no": 4, "missing": 4 , "children": [
{ "nodeid": 3, "leaf": -0.381696463 },
{ "nodeid": 4, "depth": 2, "split": "f656", "split_condition": 104.199997, "yes": 5, "no": 6, "missing": 6 , "children": [
{ "nodeid": 5, "leaf": -0.073965922 },
{ "nodeid": 6, "leaf": 0.385791391 }
]}
]}
]},
{ "nodeid": 0, "depth": 0, "split": "f284", "split_condition": 101.400002, "yes": 1, "no": 2, "missing": 2 , "children": [
{ "nodeid": 1, "leaf": -0.347005308 },
{ "nodeid": 2, "depth": 1, "split": "f300", "split_condition": 95.5, "yes": 3, "no": 4, "missing": 4 , "children": [
{ "nodeid": 3, "leaf": 0.381696492 },
{ "nodeid": 4, "depth": 2, "split": "f656", "split_condition": 104.199997, "yes": 5, "no": 6, "missing": 6 , "children": [
{ "nodeid": 5, "leaf": 0.0739660487 },
{ "nodeid": 6, "leaf": -0.385791391 }
]}
]}
]},
{ "nodeid": 0, "depth": 0, "split": "f743", "split_condition": 90.1333313, "yes": 1, "no": 2, "missing": 2 , "children": [
{ "nodeid": 1, "leaf": 0.279968023 },
{ "nodeid": 2, "depth": 1, "split": "f290", "split_condition": 108.033333, "yes": 3, "no": 4, "missing": 4 , "children": [
{ "nodeid": 3, "leaf": -0.234077886 },
{ "nodeid": 4, "leaf": 0.147569537 }
]}
]},
{ "nodeid": 0, "depth": 0, "split": "f743", "split_condition": 90.1333313, "yes": 1, "no": 2, "missing": 2 , "children": [
{ "nodeid": 1, "leaf": -0.279968172 },
{ "nodeid": 2, "depth": 1, "split": "f290", "split_condition": 108.033333, "yes": 3, "no": 4, "missing": 4 , "children": [
{ "nodeid": 3, "leaf": 0.234077781 },
{ "nodeid": 4, "leaf": -0.147569537 }
]}
]},
{ "nodeid": 0, "depth": 0, "split": "f633", "split_condition": 100.066666, "yes": 1, "no": 2, "missing": 2 , "children": [
{ "nodeid": 1, "leaf": 0.197402045 },
{ "nodeid": 2, "leaf": -0.139941201 }
]},
{ "nodeid": 0, "depth": 0, "split": "f633", "split_condition": 100.066666, "yes": 1, "no": 2, "missing": 2 , "children": [
{ "nodeid": 1, "leaf": -0.197402418 },
{ "nodeid": 2, "leaf": 0.13994123 }
]},
{ "nodeid": 0, "depth": 0, "split": "f53", "split_condition": 127.416664, "yes": 1, "no": 2, "missing": 2 , "children": [
{ "nodeid": 1, "leaf": 0.166276604 },
{ "nodeid": 2, "leaf": -0.126794532 }
]},
{ "nodeid": 0, "depth": 0, "split": "f53", "split_condition": 127.416664, "yes": 1, "no": 2, "missing": 2 , "children": [
{ "nodeid": 1, "leaf": -0.166276589 },
{ "nodeid": 2, "leaf": 0.126794517 }
]},
{ "nodeid": 0, "depth": 0, "split": "f712", "split_condition": 92.8833313, "yes": 1, "no": 2, "missing": 2 , "children": [
{ "nodeid": 1, "leaf": -0.123596296 },
{ "nodeid": 2, "leaf": 0.151612476 }
]},
{ "nodeid": 0, "depth": 0, "split": "f712", "split_condition": 92.8833313, "yes": 1, "no": 2, "missing": 2 , "children": [
{ "nodeid": 1, "leaf": 0.123594895 },
{ "nodeid": 2, "leaf": -0.151612416 }
]},
{ "nodeid": 0, "depth": 0, "split": "f633", "split_condition": 100.066666, "yes": 1, "no": 2, "missing": 2 , "children": [
{ "nodeid": 1, "leaf": 0.113635384 },
{ "nodeid": 2, "leaf": -0.107038438 }
]},
{ "nodeid": 0, "depth": 0, "split": "f633", "split_condition": 100.066666, "yes": 1, "no": 2, "missing": 2 , "children": [
{ "nodeid": 1, "leaf": -0.11363478 },
{ "nodeid": 2, "leaf": 0.107038461 }
]},
{ "nodeid": 0, "depth": 0, "split": "f602", "split_condition": 92.8499985, "yes": 1, "no": 2, "missing": 2 , "children": [
{ "nodeid": 1, "leaf": -0.1141828 },
{ "nodeid": 2, "leaf": 0.110286467 }
]},
{ "nodeid": 0, "depth": 0, "split": "f602", "split_condition": 92.8499985, "yes": 1, "no": 2, "missing": 2 , "children": [
{ "nodeid": 1, "leaf": 0.114181779 },
{ "nodeid": 2, "leaf": -0.110286497 }
]}
]
16 changes: 9 additions & 7 deletions go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ go 1.21.7
toolchain go1.21.12

require (
github.com/Elvenson/xgboost-go v0.1.4
github.com/disintegration/imaging v1.6.2
github.com/pkg/errors v0.9.1
go.viam.com/rdk v0.28.0
go.viam.com/test v1.1.1-0.20220913152726-5da9916c08a2
Expand All @@ -31,10 +33,10 @@ require (
github.com/campoy/embedmd v1.0.0 // indirect
github.com/cenkalti/backoff v2.2.1+incompatible // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
github.com/chewxy/math32 v1.10.1 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.2.0 // indirect
github.com/desertbit/timer v0.0.0-20180107155436-c41aec40b27f // indirect
github.com/disintegration/imaging v1.6.2 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/edaniels/golog v0.0.0-20230215213219-28954395e8d0 // indirect
github.com/edaniels/lidario v0.0.0-20220607182921-5879aa7b96dd // indirect
Expand All @@ -59,7 +61,7 @@ require (
github.com/golang/snappy v0.0.4 // indirect
github.com/gonuts/binary v0.2.0 // indirect
github.com/google/flatbuffers v2.0.8+incompatible // indirect
github.com/google/go-cmp v0.5.9 // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/s2a-go v0.1.4 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.2.3 // indirect
Expand Down Expand Up @@ -138,14 +140,14 @@ require (
golang.org/x/arch v0.3.0 // indirect
golang.org/x/crypto v0.23.0 // indirect
golang.org/x/exp v0.0.0-20230725012225-302865e7556b // indirect
golang.org/x/image v0.15.0 // indirect
golang.org/x/mod v0.14.0 // indirect
golang.org/x/image v0.18.0 // indirect
golang.org/x/mod v0.17.0 // indirect
golang.org/x/net v0.25.0 // indirect
golang.org/x/oauth2 v0.10.0 // indirect
golang.org/x/sync v0.6.0 // indirect
golang.org/x/sync v0.7.0 // indirect
golang.org/x/sys v0.20.0 // indirect
golang.org/x/text v0.15.0 // indirect
golang.org/x/tools v0.17.0 // indirect
golang.org/x/text v0.16.0 // indirect
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
gonum.org/v1/gonum v0.12.0 // indirect
gonum.org/v1/plot v0.12.0 // indirect
Expand Down
Loading

0 comments on commit 1c90b5c

Please sign in to comment.