Not a bug, but a question

[linuxgurugamer]

I'm wondering if this will work:

Problem: I need to release a mod for both 1.7.3 and 1.8. The mod is already built and release for 1.8

I'm thinking of releasing the mod for 1.7 using the .version file. Then, after it's been indexed, to do another release, bumping the version number, for 1.8

I don't know how CKAN will deal with this.

[HebaruSan]

Just release them and make sure the version info is correct, everything should work out fine.

And please don't do #7447 again. Game version numbers should be separate from identifiers.

[linuxgurugamer]

Sorry, that was a mistake

[linuxgurugamer]

Just release them and make sure the version info is correct, everything should work out fine.

Ok, I did the first release of BetterTimeWarpCont, version 2.3.11.7 for KSP 1.6.1-1.7.3. it's in Spacedock now
The .netkan is pointing to Spacedock.

As of now, CKAN has not updated with it

[DasSkelett]

http://status.ksp-ckan.space/
Too many .version files located: GameData/BetterTimeWarp/BetterTimeWarp.version, GameData/BetterTimeWarp/BetterTimeWarp.173.version

[linuxgurugamer]

If this is the same issue as before, then how should we name the zip files to differentiate the game version?
@DasSkelett Fixed that a few minutes ago, only one in the current file

[linuxgurugamer]

Oh yuck, not again

[linuxgurugamer]

Spacedock seems to keep pulling up a file that I deleted, so I'm going to have to do yet another version bump

[linuxgurugamer]

Ok, uploaded new file, downloaded it to verify, ran netkan.exe locally to verify no error

[DasSkelett]

KSP-CKAN/CKAN-meta@fc769d6
2.3.11.8 is now indexed, looks about right.

[linuxgurugamer]

Thank you.
Why did it i take so long, is the interface with Spacedock broken?

[DasSkelett]

Not sure. The webhook didn't trigger anymore after the second upload for some reason, even for the new release.
Just the normal bot pass picked it up...

[linuxgurugamer]

Ok, that explains a lot.
Thank you.
I just uploaded the 1.8 version of BTW to spacedock, I verified that the file only had one .version, so hopefully it will be fine

[techman83]

The webhooks got caught up with the Indexer issue, so it would have caught up when I forced a full index run.

[DasSkelett]

The webhooks got caught up with the Indexer issue, so it would have caught up when I forced a full index run.

There were already several normal passes between your forced full run and when @linuxgurugamer uploaded the new version the first run.

Lemme try to list the events up for easier understanding:

0. Indexer issue > fix merged > force full run
1. Several normal bot passes in between (~7 hours long window, so probably 2-3 passes). I can confirm this because I checked whether they did go all right after the fix was merged.
2. LGG created a new mod release (he wrote this comment), version 2.3.11.7: webhook worked, but two .version files > Inflation failed
3. LGG replaced the zip of this release with one with only a single .version file, however SpaceDock has some bug there which resulted in the first releases zip being kept, but still triggered the webhook. Inflation failed again because still 2 .version files thanks to SD bug.
   I can confirm there was another inflation attempt, because the inflation time on the status page changed.
4. LGG drafted a new release, version 2.3.11.8: webhook didn't trigger anymore
5. LGG did a "fake" change to the netkan file and merged it into the netkan repo: neither did this trigger the webhook
6. About one hour later, a normal bot pass started, which picked up the new release on SpaceDock.

So somehow the webhooks stopped working for a short time again. Multiple webhooks should've been triggered (SD update, GH NetKAN Merge), none did.

[techman83]

The normal bot pass wouldn't have picked up the SD, I haven't finished the code I was working on to do regular full passes. The key here is, webhooks now go into the regular inflation queue. Now I think we may want to look at how we address that, but I'm hesitant to add yet another container constantly polling a queue that on average gets a few hits per day. Memory isn't free and the queues are only free until 1 million requests.

As an aside, I really don't like fake changes to trigger actions and IMO they should be avoided (they pollute the commit history). If they're required, there is a failure in our processes and we should look at how we address them.

[DasSkelett]

The normal bot pass wouldn't have picked up the SD

It would have (and it did), because this NetKAN has a vref, and we include netkans with vref in normal bot passes.

The key here is, webhooks now go into the regular inflation queue.

Yes I know that, but since the inflator still runs 24/7 until we implement KSP-CKAN/NetKAN-Infra#14, webhook-submitted inflation requests are pulled and executed pretty quick normally, especially if the queue is currently empty otherwise.

there is a failure in our processes and we should look at how we address them.

That's what I wanted to say, I suspect there's another bug hidden somewhere, or the message duplication prevention kicked in, or something else. It's hard to say what part failed, the webhooks could have worked perfectly, but weren't able to submit the message to the queue maybe, who knows.

[techman83]

It would have (and it did), because this NetKAN has a vref, and we include netkans with vref in normal bot passes.

Aha. Missed that!

Yes I know that, but since the inflator still runs 24/7 until we implement KSP-CKAN/NetKAN-Infra#14, webhook-submitted inflation requests are pulled and executed pretty quick normally, especially if the queue is currently empty otherwise

It can be up to 45 minutes though, if an indexing run had just started.

That's what I wanted to say, I suspect there's another bug hidden somewhere, or the message duplication prevention kicked in, or something else. It's hard to say what part failed, the webhooks could have worked perfectly, but weren't able to submit the message to the queue maybe, who knows.

If the message was received at the start of the run, it's possible it got de-duplicated. But it's immaterial at that point.

[HebaruSan]

The key here is, webhooks now go into the regular inflation queue. Now I think we may want to look at how we address that, but I'm hesitant to add yet another container constantly polling a queue that on average gets a few hits per day.

It can be up to 45 minutes though, if an indexing run had just started.

Maybe we should throttle the scheduler somehow? If it submitted say 30 mods and then waited till the queue was empty or near-empty to submit 30 more, webhook inflations could happen in between those batches without having to wait so long.

[techman83]

Problem is kinda similar, the scheduler will make a lot of unnecessary API calls figuring out if it can send more 30 could take some time to be inflated or be really quick, leading to wondering if it's working or not. We could have a second inlfator launch, have it timeout after X minutes of no new netkan inflations and have a separate queue. We could even use that for the main scheduler and just have it do the same launch process. Aka KSP-CKAN/NetKAN-Infra#14

[linuxgurugamer]

As an aside, I really don't like fake changes to trigger actions and IMO they should be avoided (they pollute the commit history). If they're required, there is a failure in our processes and we should look at how we address them

I fully understand. It would not be needed iff there was a way to trigger actions without a fake change. Is there any way to do that?

[techman83]

I fully understand. It would not be needed iff there was a way to trigger actions without a fake change. Is there any way to do that?

Well, most mod authors don't have the ability to merge their own PRs. So normally it's a case of wait until the next bot run.

The new infrastructure could potentially allow for a process to trigger inflations, but creating a secure interface for it has been hard. Maybe we can get a quick win with a discord bot? /inflate ModIdentifier

[DasSkelett]

Maybe we can get a quick win with a discord bot? /inflate ModIdentifier

Cool idea. We could use the netkan-bot channel for that (once it is fully set up to allow writing messages in it, and limited to only certain groups one of which would include @linuxgurugamer)