chore(web): add image-based lighting

[airslice]

Overview

Supports image-based lighting for beta.

What I've done

Add image based lighting properties to scene, which should effect PBR models & 3d tiles in scene:

type SceneProperty = {
  light?: {
    // ...
    specularEnvironmentMaps?: string, // URL to a KTX2 format
    sphericalHarmonicCoefficients?: [x: number, y: number, z: number][],
    imageBasedLightIntensity?: number, // Default: 1
  };
};

Add image based lighting properties to 3d tiles appearance & model appearance

type Cesium3DTilesAppearance = {
  // ...
  specularEnvironmentMaps?: string;
  sphericalHarmonicCoefficients?: [x: number, y: number, z: number][];
  imageBasedLightIntensity?: number;
}

type ModelAppearance = {
  // ...
  specularEnvironmentMaps?: string;
  sphericalHarmonicCoefficients?: [x: number, y: number, z: number][];
  imageBasedLightIntensity?: number;
}

What I haven't done

How I tested

Scene IBL

let layerId = reearth.layers.add({
  type: "simple",
  data: {
    type: "3dtiles",
    url: "https://assets.cms.plateau.reearth.io/assets/11/6d05db-ed47-4f88-b565-9eb385b1ebb0/13100_tokyo23-ku_2022_3dtiles%20_1_1_op_bldg_13101_chiyoda-ku_lod1/tileset.json",
  },
  "3dtiles": {
  }
});
setTimeout(()=>{
  reearth.camera.flyTo(layerId, {duration: 0});
},10);

reearth.scene.overrideProperty({
    light: {
      sphericalHarmonicCoefficients: [
        [0.499745965003967, 0.499196201562881, 0.500154078006744],
        [0.265826553106308, -0.266099184751511, 0.265922993421555],
        [0.243236944079399, 0.266723394393921, -0.265380442142487],
        [-0.266895800828934, 0.265416264533997, 0.266921550035477],
        [0.000195000306121, -0.000644546060357, -0.000383183418307],
        [-0.000396036746679, -0.000622032093816, 0.000262127199676],
        [-0.000214280473301, 0.00004872302452, -0.000059724134189],
        [0.000107143961941, -0.000126510843984, -0.000425444566645],
        [-0.000069071611506, 0.000134039684781, -0.000119135256682],
      ],
      imageBasedLightIntensity: 1,
    },
});

reearth.scene.overrideProperty({
    light: {
      specularEnvironmentMaps: "https://cesium.com/public/SandcastleSampleData/kiara_6_afternoon_2k_ibl.ktx2"
    },
  });

reearth.scene.overrideProperty({
    light: {
    },
});

3d tiles IBL

let layerId2 = reearth.layers.add({
  type: "simple",
  data: {
    type: "3dtiles",
    url: "https://assets.cms.plateau.reearth.io/assets/5d/134c5b-d8a0-4cca-aeff-2354188a17ca/13100_tokyo23-ku_2022_3dtiles%20_1_1_op_bldg_13102_chuo-ku_lod1/tileset.json",
  },
  "3dtiles": {}
});
setTimeout(()=>{
  reearth.camera.flyTo(layerId2, {duration: 0});
},10);

reearth.layers.override(layerId2, {
  "3dtiles": {
    sphericalHarmonicCoefficients: [
        [0.499745965003967, 0.499196201562881, 0.500154078006744],
        [0.265826553106308, -0.266099184751511, 0.265922993421555],
        [0.243236944079399, 0.266723394393921, -0.265380442142487],
        [-0.266895800828934, 0.265416264533997, 0.266921550035477],
        [0.000195000306121, -0.000644546060357, -0.000383183418307],
        [-0.000396036746679, -0.000622032093816, 0.000262127199676],
        [-0.000214280473301, 0.00004872302452, -0.000059724134189],
        [0.000107143961941, -0.000126510843984, -0.000425444566645],
        [-0.000069071611506, 0.000134039684781, -0.000119135256682],
      ],
      imageBasedLightIntensity: 1,
  },
});

reearth.layers.override(layerId2, {
  "3dtiles": {
    sphericalHarmonicCoefficients: [],
  },
});

reearth.layers.override(layerId2, {
  "3dtiles": {
    sphericalHarmonicCoefficients: [],
    specularEnvironmentMaps: "https://cesium.com/public/SandcastleSampleData/kiara_6_afternoon_2k_ibl.ktx2"
  },
});

Model IBL

const modelLayerId = reearth.layers.add({
  type: "simple",
  data: {
    type: "geojson",
    value: {
      type: "Feature",
      geometry: {
        type: "Point",
        coordinates: [139.753214, 35.684891, 1], // Japan Tokyo Chiyoda-ku
      },
    },
  },
  model: {
    url: "https://api.test.reearth.dev/assets/01h5478z9t8r9cshbwvpxepwdd.glb",
    scale: 1000,
  },
});
setTimeout(()=>{
  reearth.camera.flyTo(modelLayerId, {duration: 0});
},10);

reearth.layers.override(modelLayerId, {
  model: {
    sphericalHarmonicCoefficients: [
        [0.499745965003967, 0.499196201562881, 0.500154078006744],
        [0.265826553106308, -0.266099184751511, 0.265922993421555],
        [0.243236944079399, 0.266723394393921, -0.265380442142487],
        [-0.266895800828934, 0.265416264533997, 0.266921550035477],
        [0.000195000306121, -0.000644546060357, -0.000383183418307],
        [-0.000396036746679, -0.000622032093816, 0.000262127199676],
        [-0.000214280473301, 0.00004872302452, -0.000059724134189],
        [0.000107143961941, -0.000126510843984, -0.000425444566645],
        [-0.000069071611506, 0.000134039684781, -0.000119135256682],
      ],
    imageBasedLightIntensity: 0.5,
    specularEnvironmentMaps: '',
  },
});

reearth.layers.override(modelLayerId, {
  model: {
    sphericalHarmonicCoefficients: '',
    specularEnvironmentMaps: "https://cesium.com/public/SandcastleSampleData/kiara_6_afternoon_2k_ibl.ktx2",
  },
});

reearth.layers.override(modelLayerId, {
  model: {
    sphericalHarmonicCoefficients: [],
    specularEnvironmentMaps: '',
  },
});

Which point I want you to review particularly

Memo

• Our override rule seems different on scene property and layer property.
• Override scene property:
  • undefined can be used.
  • overridden property will replace the previous value.
• Override layer property:
  • undefined will be ignored.
  • overridden property will be merged with the previous value.

[netlify]

✅ Deploy Preview for reearth-web canceled.

Name	Link
🔨 Latest commit	50e9751
🔍 Latest deploy log	https://app.netlify.com/sites/reearth-web/deploys/64b73e6daddaea0008ff257a

[codecov]

Codecov Report

Merging #519 (50e9751) into main (9473a5a) will increase coverage by 0.00%.
The diff coverage is 26.63%.

Additional details and impacted files

@@           Coverage Diff            @@
##             main     #519    +/-   ##
========================================
  Coverage   28.48%   28.49%            
========================================
  Files        1392     1392            
  Lines      150120   150308   +188     
  Branches     3591     3590     -1     
========================================
+ Hits        42769    42837    +68     
- Misses     106199   106319   +120     
  Partials     1152     1152            

Flag	Coverage Δ
web	26.40% <26.63%> (+0.01%)	⬆️
web-beta	26.40% <26.63%> (+0.01%)	⬆️
web-classic	26.40% <26.63%> (+0.01%)	⬆️
web-utils	26.40% <26.63%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
web/src/beta/lib/core/Map/types/index.ts	0.00% <0.00%> (ø)
.../lib/core/engines/Cesium/Feature/Tileset/index.tsx	25.26% <0.00%> (-0.55%)	⬇️
...src/beta/lib/core/engines/Cesium/Feature/index.tsx	33.01% <0.00%> (-3.64%)	⬇️
...ta/lib/core/engines/Cesium/Feature/Model/index.tsx	21.07% <23.46%> (+0.14%)	⬆️
...a/lib/core/engines/Cesium/Feature/Tileset/hooks.ts	17.72% <23.63%> (+0.61%)	⬆️
web/src/beta/lib/core/Map/Layer/index.tsx	100.00% <100.00%> (ø)
...src/beta/lib/core/engines/Cesium/Feature/utils.tsx	67.47% <100.00%> (+0.81%)	⬆️
...src/beta/lib/core/mantle/evaluator/simple/index.ts	77.27% <100.00%> (ø)
web/src/beta/lib/core/mantle/types/appearance.ts	100.00% <100.00%> (ø)

... and 1 file with indirect coverage changes