git-subrepo: Can't commit: 'subrepo/myrepo' doesn't contain upstream HEAD:

[Lindsay-Mathieson]

Getting the above error when trying to execute
git subrepo push myrepo

"myrepo" is on the master branch, the parent repository on development branch

[admorgan]

That means the SHA in the commit field of your .gitrepo file isn't a valid SHA on the remote repo. This can happen if the remote's history was rewritten. Rewriting the history of a public repo is extremely dangerous so I would be wary if you can't identify what happened.

[Lindsay-Mathieson]

That means the SHA in the commit field of your .gitrepo file isn't a valid SHA on the remote repo. This can happen if the remote's history was rewritten. Rewriting the history of a public repo is extremely dangerous so I would be wary if you can't identify what happened.

Thanks, good to know

[Lindsay-Mathieson]

That means the SHA in the commit field of your .gitrepo file isn't a valid SHA on the remote repo. This can happen if the remote's history was rewritten. Rewriting the history of a public repo is extremely dangerous so I would be wary if you can't identify what happened.

Well, I ended up removing myrepo and recloning it using subrepo, almost immediately ran into the same problem. Ended up running

subrepo push -f myrepo master

It overwrote myrepo with the parent repository! I lost all history and code. Thank god I had a backup.

So far subrepo has been a fragile disaster for me. Only useful if I only make/push changes as a std repository, then pull them projects that have it as a subrepo. Edit/push via the subrepo just seem to break regularly.

[admorgan]

I am really curious about your use case. We use subrepo daily in a corporate environment and have not experienced such fragility. Are there any examples you have of public repos you are experiencing this on? Is there any way you can generate logs that you would be willing to share so I can help alleviate this for you?

[Lindsay-Mathieson]

Are there any examples you have of public repos you are experiencing this on? Is there any way you can generate logs that you would be willing to share so I can help alleviate this for you?

Unfortunately, I'm working on private repos (github) for a private company. But logs shouldn't be a problem, though I'm not sure what logs would be useful.

I'm working in Visual Studio and wondering if the problem is I make changes to the parent project and the subrepo project and do commits from there, before dropping to a command line to do the subrepo push - would that mess things up.

My apologies for my tone earlier, it was unwarranted. Been a long day here.

[admorgan]

The Visual Studio should not inject such a problem since that is also my workflow. If you decide to keep using subrepo, next time you try to do a push could you do a git subrepo -v push That will show us exactly what subrepo is doing and allow me to refine further investigation into the matter.

[Lindsay-Mathieson]

The Visual Studio should not inject such a problem since that is also my workflow. If you decide to keep using subrepo, next time you try to do a push could you do a git subrepo -v push That will show us exactly what subrepo is doing and allow me to refine further investigation into the matter.

Will do

[mmcev106]

@admorgan, I've been seeing this intermittently for the better part of a year now. I've been trying to track it down, but may need help in order to get to the bottom of it. Verbose output can be found here. The subrepo can be found here. The containing repo is private. I've been trying to hack the git-subrepo source to track it down. Here is the command that is failing unexpectedly:

git rev-list 'subrepo/docs' | grep -q '^5d843c18f26e11b8d5c611696d871495f2065a3e'

I believe the specified commit is not contained in the rev-list because it is getting rebased unexpectedly by this line. I conclude this because I would expected git log --oneline refs/subrepo/docs/branch to show this:

...
5d843c1 Update crons.md
29ec5a2 Update guide.md
5078b55 Manually added a version since subrepo push failed
...

...but instead it shows this, which of course means 5d843c1 won't be found:

...
880cd99b Update crons.md
02bd1054 Update guide.md
5078b55b Manually added a version since subrepo push failed
...

I should be able to reproduce this consistently by creating test repos from these commits, but am at a loss as to how to solve it. At the moment, it seems like using filter-branch here is simply unsafe, but I just don't fully understand the context. How can I help pinpoint the issue and/or implement a solution?

[mmcev106]

@admorgan, I just did a force push to get around & keep moving (in case you test the docs repo in its current state). If example is useful, I could recreate the branches necessary to reproduce this.

[mmcev106]

@admorgan, it happened again and I found simple way for anyone to reproduce the issue by running the following commands:

mkdir test-repo
cd test-repo
git init
touch test-file-1
git add test-file-1
git commit -m 'Added test-file-1'
git subrepo clone [email protected]:vanderbilt-redcap/external-module-framework-docs.git -b git-subrepo-bug-1 docs
touch docs/test-file-2
git add docs/test-file-2
git commit -m 'Added docs/test-file-2'
git subrepo pull docs -b git-subrepo-bug-2
git subrepo push docs

[taktran]

I found a potential fix for this issue here: #617

[admorgan]

There are two elements at play here. The technical element about why this doesn't work, and the UI issue where the report doesn't do a good enough job describing it.

The technical explanation

All of subrepo's state is stored in the .gitrepo file that makes a subdirectory a subrepo.
Lets look at how that file is modified based on the actions in your excellent example.

1. git subrepo init
   1. branch = git-subrepo-bug-1
   2. commit = SHA of tips of git-subrepo-bug-1
2. git subrepo pull -b
   1. commit = SHA of tips of git-subrepo-bug-2

The problem is that there is a SHA that doesn't exist on git-subrepo-but-1, and subrepo can't do any more actions.

The UI issue

1. Subrepo should never knowingly put it self in a situation where it can't work any longer
   • Options
     1. Error subrepo pull if the SHA update is not reachable by the branch in the .gitrepo
        • Explain the branch doesn't have that SHA
        • Suggest using -u to update the following branch
     2. subrepo pull should automatically record the branch change
        • Currently handled by -u
        • Would need an anti -u because some edge cases would be harmed by a branch change
2. Better error message if the upstream gets rewritten
   3. Search for branches with the SHA and suggest them
   4. How to describe a completely missing SHA

[mmcev106]

@admorgan, it seems like you may have missed important details from my previous messages. The SHAs in .gitrepo are always present upstream. The unexpected SHA is being created by git-subrepo on the git-filter-branch line I linked to. In other words, git-subrepo is knowingly creating the problem. I think we might be running into one of the plethora of pitfalls mentioned in the git-filter-branch docs. I tried digging into it to a few times & replacing that line with an alternate solution that didn't effectively cause a rebase, but it's over my head for the time I've had available so far.

[admorgan]

@mmcev106 sorry about not listening close enough. You are correct the issues I list are not related to the issue you are having. The issue you are having is because the parent value did not get updated correctly in the git pull. I will try to find some time to dig into deeply next week.

[admorgan]

Just want you to know that I am still investigating this, just haven't had as much time as I need.

[mmcev106]

I appreciate you investigating it! I know it's not exactly trivial.

[n3dst4]

I am having the same issue: git subrepo pull... always works, but git subrepo push... always fails because the local branch created by subrepo does not contain the commit id in .gitrepo. Neither the subrepo nor the containing/parent repo have been rebased or rewritten.

I have found that the --squash flag to git subrepo push ... is a workaround. It means you get a squashed commit in the subrepo's history, but that's acceptable for my workflow.

I second the sentiment above - I really appreciate what you are doing with git-subrepo and I look forward to a real fix, but I also get what a chore a successful open source project like this can be. Thank you.

[n3dst4]

If you get time to work on this, feel free to contact me for repo links, logs, etc.