Unable to download or change Tonie content #5

eSKape-debug · 2024-11-05T15:05:51Z

Problem/Motivation

After successful flashing and adding a CC2300 Box to Teddycloud (running as this HA Addon Container on HAOS on an RPi 4) i wanted to verify, that an original Tonie (that is playable, i think only because it was already downloaded and played pre-flashing) gets automatically downloaded and cached inside Teddycloud (which enables the possibility to play it inside the browser.

Expected behavior

Download starts automatically and after it finishes, I am able to play it in the browser.

Actual behavior

Download started but after a second, this error is shown:

Additionally, when trying to change the source of a model to a radio stream, the following error happens:

with the following addon logs produced:

WARN |platform_linux.c:0292:socketReceive()| buffer does not contain null terminator

WARN |platform_linux.c:0292:socketReceive()| buffer does not contain null terminator

WARN |platform_linux.c:0292:socketReceive()| buffer does not contain null terminator

DEBUG|server.c:0284:httpServerRequestCallback()| No certificate authentication

DEBUG|server.c:0287:httpServerRequestCallback()| Started server request to /api/getTagInfo, request 2, by unknown/web

DEBUG|server.c:0289:httpServerRequestCallback()|  >> client requested '/api/getTagInfo' via GET 
DEBUG|server.c:0534:httpServerRequestCallback()| Stopped server request to /api/getTagInfo, request 2

2024/11/05 15:51:56 [error] 272#272: *124 connect() failed (111: Connection refused) while connecting to upstream, client: 172.30.32.2, server: ff7725ba-teddycloud, request: "GET /api/getTagInfo?ruid=c62ca116500304e0?overlay%3D10082C71290C HTTP/1.1", upstream: "http://[::1]:80/api/getTagInfo?ruid=c62ca116500304e0?overlay%3D10082C71290C", host: "192.168.178.34:8123", referrer: "http://192.168.178.34:8123/api/hassio_ingress/exC2FbRGkw-5V6_UBCRYiiPJj7BD0gzlb4kD_2DU2ps/web/tonies"
WARN |platform_linux.c:0292:socketReceive()| buffer does not contain null terminator

WARN |platform_linux.c:0292:socketReceive()| buffer does not contain null terminator

WARN |platform_linux.c:0292:socketReceive()| buffer does not contain null terminator

WARN |platform_linux.c:0292:socketReceive()| buffer does not contain null terminator

Logs when starting the Toniebox:

INFO |server.c:0959:server_init()| 1 open HTTPS API connections

DEBUG|server.c:0276:httpServerRequestCallback()| Certificate authentication:

DEBUG|server.c:0277:httpServerRequestCallback()|   Issuer:     'Boxine Factory SubCA 02'

DEBUG|server.c:0278:httpServerRequestCallback()|   Subject:    'b'10082C71290C''

DEBUG|server.c:0279:httpServerRequestCallback()|   Serial:     '1A48A7'

DEBUG|server.c:0287:httpServerRequestCallback()| Started server request to /v1/time, request 2, by b'10082C71290C'

DEBUG|server.c:0289:httpServerRequestCallback()|  >> client requested '/v1/time' via GET 
INFO |server.c:0430:httpServerRequestCallback()| UA=TB/1620325289 SP/34471936 HW/1048921, FW=1620325289, SP=34471936, HW=1048921

INFO |handler_cloud.c:0041:handleCloudTime()|  >> respond with current time

INFO |mqtt.c:0699:mqtt_init_box()| Registered new box 'Toniebox_Lukas' (cn: '10082C71290C')

INFO |mqtt.c:0700:mqtt_init_box()| Using base path 'teddyCloud/box/10082C71290C' and id 'teddyCloud_Box_10082C71290C'

DEBUG|home_assistant.c:0073:ha_publish()| [HA] Publish
DEBUG|home_assistant.c:0073:ha_publish()| [HA] Publish
DEBUG|server.c:0534:httpServerRequestCallback()| Stopped server request to /v1/time, request

Logs when doing a freshness check:

INFO |server.c:0959:server_init()| 2 open HTTPS API connections

DEBUG|server.c:0276:httpServerRequestCallback()| Certificate authentication:

DEBUG|server.c:0277:httpServerRequestCallback()|   Issuer:     'Boxine Factory SubCA 02'

DEBUG|server.c:0278:httpServerRequestCallback()|   Subject:    'b'10082C71290C''

DEBUG|server.c:0279:httpServerRequestCallback()|   Serial:     '1A48A7'

DEBUG|server.c:0287:httpServerRequestCallback()| Started server request to /v1/freshness-check, request 2, by b'10082C71290C'

DEBUG|server.c:0289:httpServerRequestCallback()|  >> client requested '/v1/freshness-check' via POST 
INFO |server.c:0430:httpServerRequestCallback()| UA=TB/1620325289 SP/34471936 HW/1048921, FW=1620325289, SP=34471936, HW=1048921

INFO |handler_cloud.c:0817:handleCloudFreshnessCheck()| Content (944 of 944)
INFO |handler_cloud.c:0825:handleCloudFreshnessCheck()| Found 59 tonies:
INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E0040350141673B9, nocloud: 0, live: 0, updated: 0, audioid: 59B80855 (2017-09-12 18:16:21)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E00403501429D870, nocloud: 0, live: 0, updated: 0, audioid: 646B633C (2023-05-22 14:42:36)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E0040350154901E6, nocloud: 0, live: 0, updated: 0, audioid: 59CB9CCA (2017-09-27 14:42:50)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E0040350134730AE, nocloud: 0, live: 0, updated: 0, audioid: 00000001 (special), audioid-server: 00000001 (special)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E00403501541DA58, nocloud: 0, live: 0, updated: 0, audioid: 641AC980 (2023-03-22 10:25:20)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E004035016A12CC6, nocloud: 0, live: 0, updated: 0, audioid: 5BD9B2B8 (2018-10-31 14:48:40)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E004035015A38D35, nocloud: 0, live: 0, updated: 0, audioid: 61113F95 (2021-08-09 16:45:41)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E0040350133C430C, nocloud: 0, live: 0, updated: 0, audioid: 5E5D1A11 (2020-03-02 15:37:05)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E0040350175FF0BA, nocloud: 0, live: 0, updated: 0, audioid: 6253C838 (2022-04-11 08:18:32)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E004035014DA3D5D, nocloud: 0, live: 0, updated: 0, audioid: 646B6355 (2023-05-22 14:43:01)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E004035017A66765, nocloud: 0, live: 0, updated: 0, audioid: 6253C3E3 (2022-04-11 08:00:03)

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E00403501575D2F6, nocloud: 0, live: 0, updated: 0, audioid: 64CD3D47 (2023-08-04 20:02:47)

DEBUG|contentJson.c:0254:content_json_update_model()| Audio-id 64CD3D47 unknown but previous content known by model .

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: E004035015A58B17, nocloud: 0, live: 0, updated: 0, audioid: 60339FDD (2021-02-22 13:13:17)

...
...
...
DEBUG|contentJson.c:0254:content_json_update_model()| Audio-id 57EE897A unknown but previous content known by model .

INFO |handler_cloud.c:0885:handleCloudFreshnessCheck()|   uid: 1800000001000000, nocloud: 0, live: 0, updated: 0, audioid: 57EE897A (2016-09-30 17:49:14)

DEBUG|contentJson.c:0254:content_json_update_model()| Audio-id 57EE897A unknown but previous content known by model .

INFO |cloud_request.c:0200:web_request()| Connecting to HTTP server prod.de.tbs.toys:443...

INFO |cloud_request.c:0252:web_request()|   trying IP: 3.74.99.150
DEBUG|cloud_request.c:0033:httpClientTlsInitCallbackBase()| Initializing TLS...

DEBUG|cloud_request.c:0070:httpClientTlsInitCallbackBase()| Initializing TLS done

Steps to reproduce

Flash CC2300 Box as described inside teddycloud step-by-step
add Box to teddycloud
The "Boxine" cloud connection Icon on the top right shows green.
put an original Tonie on the box (was previously downloaded already)
Click on the Download button in the middle

Here is my config:

configVersion=13
log.level=5
log.color=true
log.logFullAuth=true
core.server.http_port=80
core.server.https_web_port=8443
core.server.https_api_port=443
core.server.bind_ip=
core.host_url=http://localhost
core.certdir=certs/client
core.contentdir=default
core.librarydir=library
core.datadir=data
core.firmwaredir=firmware
core.cachedir=cache
core.sslkeylogfile=
core.new_webgui_as_default=true
core.server_cert.file.ca=certs/server/ca-root.pem
core.server_cert.file.ca_der=certs/server/ca.der
core.server_cert.file.ca_key=certs/server/ca-key.pem
core.server_cert.file.crt=certs/server/teddy-cert.pem
core.server_cert.file.key=certs/server/teddy-key.pem
core.server_cert.data.ca=
core.client_cert.file.ca=certs/client/ca.der
core.client_cert.file.crt=certs/client/client.der
core.client_cert.file.key=certs/client/private.der
core.allowOrigin=
core.boxCertAuth=true
core.allowNewBox=false
core.flex_enabled=true
core.flex_uid=
core.settings_level=3
core.tonies_json_auto_update=true
security_mit.warnAccess=true
security_mit.lockAccess=true
security_mit.httpsOnly=true
security_mit.onBlacklistDomain=true
security_mit.onCrawler=true
security_mit.onRobotsTxt=true
cloud.enabled=true
cloud.remote_hostname=prod.de.tbs.toys
cloud.remote_port=443
cloud.enableV1Claim=true
cloud.enableV1CloudReset=false
cloud.enableV1FreshnessCheck=true
cloud.enableV1Log=false
cloud.enableV1Time=false
cloud.enableV1Ota=false
cloud.enableV2Content=true
cloud.cacheOta=true
cloud.localOta=false
cloud.cacheContent=true
cloud.cacheToLibrary=true
cloud.markCustomTagByPass=true
cloud.prioCustomContent=true
cloud.updateOnLowerAudioId=true
cloud.dumpRuidAuthContentJson=true
encode.bitrate=96
encode.ffmpeg_stream_buffer_ms=2000
encode.ffmpeg_stream_restart=false
encode.ffmpeg_sweep_startup_buffer=true
encode.ffmpeg_sweep_delay_ms=2000
encode.stream_max_size=251658239
frontend.split_model_content=true
frontend.ignore_web_version_mismatch=false
frontend.confirm_audioplayer_close=true
toniebox.api_access=true
toniebox.overrideCloud=true
toniebox.max_vol_spk=3
toniebox.max_vol_hdp=3
toniebox.slap_enabled=true
toniebox.slap_back_left=false
toniebox.led=0
rtnl.logRaw=false
rtnl.logHuman=true
rtnl.logRawFile=config/rtnl.bin
rtnl.logHumanFile=config/rtnl.csv
pcap.enabled=false
pcap.filename=config/traffic.pcap
mqtt.enabled=false
mqtt.hostname=
mqtt.port=1883
mqtt.username=
mqtt.password=
mqtt.identification=
mqtt.topic=teddyCloud
mqtt.qosLevel=0
hass.name=teddyCloud - Server
hass.id=teddyCloud_Server
tonie_json.cache_images=true
tonie_json.cache_preload=false
debug.web.pcm_encode_console_url=false

The content of the Homeassistant addon dir:

The ngCfg.json on SD-Card:

{
    "general": {
        "activeImg": "ofw2",
        "_descWaitForPress": "Waits for an earpress on startup",
        "waitForPress": false,
        "_descWaitForBoot": "Waits for an earpress before firmware boot",
        "waitForBoot": false,
        "waitTimeoutInS": 60,
        "_descMinBatteryLevel": "Divide through 2785 to get voltage",
        "minBatteryLevel": 8869,
        "ofwFixFlash": "/sys/pre-img.bin",
        "_descSerialLog": "Logging only works with the debug build!",
        "serialLog": false,
        "_descLogLevel": "0:Trace - 5:Fatal",
        "logLevel": 0,
        "_descLogColor": "Use colors in log output",
        "logColor": false
    },
    "ofw1": {
        "checkHash": false,
        "hashFile": false,
        "watchdog": true,
        "bootFlashImg": true,
        "flashImg": "/sys/pre-img.bin"
    },
    "ofw2": {
        "checkHash": true,
        "hashFile": false,
        "watchdog": true,
        "ofwFix": true,
        "ofwSimBL": true,
        "patches": ["altCa.305", "altUrl.custom.305"]
    },
    "ofw3": {
        "checkHash": true,
        "hashFile": false,
        "watchdog": true,
        "ofwFix": true,
        "patches": ["altCa.305", "altUrl.tc.fritz.box"]
    },
    "cfw1": {
        "checkHash": false,
        "hashFile": false,
        "watchdog": true
    },
    "cfw2": {
        "checkHash": false,
        "hashFile": false,
        "watchdog": true
    },
    "cfw3": {
        "checkHash": false,
        "hashFile": false,
        "watchdog": true
    },
    "add1": {
        "checkHash": true,
        "hashFile": false,
        "watchdog": true,
        "ofwFix": true,
        "ofwSimBL": false,        
        "patches": ["blockCheck.310", "noCerts.305", "noPass3.310", "noPrivacy.305", "uidCheck.307"]
    },
    "add2": {
        "checkHash": true,
        "hashFile": false,
        "watchdog": true,
        "ofwFix": true,
        "ofwSimBL": false,
        "patches": ["altCa.305", "altUrl.305"]
    },
    "add3": {
        "checkHash": true,
        "hashFile": false,
        "watchdog": true,
        "ofwFix": true,
        "ofwSimBL": false,
        "patches": ["altCa.305", "altUrl.305"]
    }
}

I double checked, that the ca.der inside teddycloud client dir is the one from the Toniebox backup.

Proposed changes

I have no current lead - however - the connection to the Boxine cloud server does not seem to work.

The text was updated successfully, but these errors were encountered:

mrueg · 2024-11-05T19:38:52Z

Thanks for the excellent bug report!
If you want to debug it further can you try open up the "Web Developer Tools" (this is how it's called in Firefox) in your browser? I think this might fail as the javascript requests something that isn't rewritten properly by nginx in https://github.com/mrueg/addon-teddycloud/blob/main/teddycloud/rootfs/etc/nginx/servers/ingress.conf#L19 and thus sending the request to a bad endpoint.

eSKape-debug · 2024-11-05T20:37:20Z

Hi mrueg
The download request looks like this:

Does the content/download/... resource needs no be rewritten to the $ingress_path/content/download... or something similar?

Maybe interesting: Inside the Teddycloud Code at the suspected content download resource https://github.com/toniebox-reverse-engineering/teddycloud/blob/9e05ef8fc4a7d66f1e38db6a444c4a0d49af991f/src/handler_api.c#L1749, there is an ToDo saying something about rewrite

mrueg · 2024-11-06T07:19:34Z

I added a few more filters to the nginx config, please try again with the edge version. :)

eSKape-debug · 2024-11-07T13:10:37Z

i installed the edge addon - copied over the certs folder from the old addon dir to the new one, uninstalled the stable addon and started the edge one -
however, it does not seem to rewrite the URL for download - as it still is:
IP:PORT/content/download/C62CA116/500304E0?overlay=10082C71290C

Setting the radio does seem to work, the POST to IP:PORT/api/hassio_ingress/ZqlUipbpq2X-KnZyoxcBlKBP8Yhen3U9pBfsJu-Qlho/content/json/set/c62ca116500304e0?overlay=10082C71290C responses 200. However, even after a freshness check, the actual Tonie content is being played.

mrueg · 2024-11-15T20:54:02Z

Probably this is missing one or two changes in the javascript. Hopefully get some time next week.

mrueg · 2024-11-16T19:52:29Z

I see this when trying to test the webradio with a freshness check.

INFO |handler_cloud.c:0590:handleCloudContent()| Serve streaming content from https://dispatcher.rndfnk.com/rbb/fritz/live/mp3/mid
[20:47:04] INFO: Service teddycloud exited with code 256 (by signal 11)
[20:47:05] INFO: Starting TeddyCloud...

looks like there's some segfault?

mrueg · 2024-11-17T07:24:51Z

I switched the base to ubuntu toniebox-reverse-engineering/teddycloud#160 seems like it's working now :) Please give it another try.

eSKape-debug · 2024-11-18T16:00:47Z

I think we are getting closer - I think the rewrites however do not work as they are intended. When trying to download a tonie, still such an URL is used e.g.

<IP:PORT>/content/download/FCB0641D/500304E0

When manipulating and resending with the ingress URL like this:

<IP:PORT>/api/hassio_ingress/ZqlUipbpq2X-KnZyoxcBlKBP8Yhen3U9pBfsJu-Qlho/content/download/FCB0641D/500304E0

The download works and a .taf file is inside the library folder.
The /content/json/set/... URL of adding a radio stream however is correctly rewritten.

(Note: i am on the latest Edge Build 0f17acb)

eSKape-debug · 2024-11-18T16:06:16Z

Just investigated the filters: The http method is missing from my descriptions - but the download (and play) of content is a GET call. I suspect, this is why there is no rewrite currently:

addon-teddycloud/teddycloud/rootfs/etc/nginx/servers/ingress.conf

Line 36 in 0f17acb

    
           sub_filter 'apiPostTeddyCloudRaw(`/content/' 'apiPostTeddyCloudRaw(`$http_x_ingress_path/content/';

mrueg · 2024-11-18T16:51:24Z

Please try again with 8c07169 or v0.3.1, I think the download should work now.

The encoder upload still has some issues, that's more on the nginx side of things I believe.

eSKape-debug · 2024-11-18T20:54:48Z

Download still does not work.
I came across the code building the download URL: https://github.com/toniebox-reverse-engineering/teddycloud/blob/9e05ef8fc4a7d66f1e38db6a444c4a0d49af991f/src/handler_api.c#L2638 which is not considering any base - or configurable path.
That is how I found your issue on the teddycloud repo, which tackles the problem (toniebox-reverse-engineering/teddycloud#245).
I think this is not an issue of this repo anymore - correct?
For now i would stick with dirty workaround of manually rewriting the download URL with DevTools.

mrueg · 2024-11-18T22:47:01Z

yes, you're right I was fixing audioUrl, and it should have been downloadTriggerUrl. af48891 this should add a filter for it as well.

If toniebox-reverse-engineering/teddycloud#245 gets fixed, we won't need all the nginx rewrites anymore.

eSKape-debug · 2024-11-19T15:12:02Z

works now - thank you very much :)

mmslakr · 2024-11-30T19:26:32Z

@mrueg, can you release that last commit to stable?

mrueg · 2024-11-30T23:11:12Z

@mmslakr v0.3.2 is released.

eSKape-debug mentioned this issue Nov 18, 2024

Add content GET filter #7

Closed

eSKape-debug closed this as completed Nov 19, 2024

kraiz mentioned this issue Nov 30, 2024

bugfix: increase specificity of downloadTriggerUrl filter #8

Merged

github-actions bot locked and limited conversation to collaborators Dec 31, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to download or change Tonie content #5

Unable to download or change Tonie content #5

eSKape-debug commented Nov 5, 2024

mrueg commented Nov 5, 2024

eSKape-debug commented Nov 5, 2024

mrueg commented Nov 6, 2024

eSKape-debug commented Nov 7, 2024 •

edited

Loading

mrueg commented Nov 15, 2024

mrueg commented Nov 16, 2024 •

edited

Loading

mrueg commented Nov 17, 2024

eSKape-debug commented Nov 18, 2024

eSKape-debug commented Nov 18, 2024

mrueg commented Nov 18, 2024

eSKape-debug commented Nov 18, 2024

mrueg commented Nov 18, 2024

eSKape-debug commented Nov 19, 2024

mmslakr commented Nov 30, 2024

mrueg commented Nov 30, 2024

Unable to download or change Tonie content #5

Unable to download or change Tonie content #5

Comments

eSKape-debug commented Nov 5, 2024

Problem/Motivation

Expected behavior

Actual behavior

Steps to reproduce

Proposed changes

mrueg commented Nov 5, 2024

eSKape-debug commented Nov 5, 2024

mrueg commented Nov 6, 2024

eSKape-debug commented Nov 7, 2024 • edited Loading

mrueg commented Nov 15, 2024

mrueg commented Nov 16, 2024 • edited Loading

mrueg commented Nov 17, 2024

eSKape-debug commented Nov 18, 2024

eSKape-debug commented Nov 18, 2024

mrueg commented Nov 18, 2024

eSKape-debug commented Nov 18, 2024

mrueg commented Nov 18, 2024

eSKape-debug commented Nov 19, 2024

mmslakr commented Nov 30, 2024

mrueg commented Nov 30, 2024

eSKape-debug commented Nov 7, 2024 •

edited

Loading

mrueg commented Nov 16, 2024 •

edited

Loading