I just tested in 15.1.2. https://github.com/upfrontIO/microschema, //github.com/upfrontIO/microschema and upfrontIO/microschema are all converted to https://github.com/upfrontIO/microschema.

Then we run git push --dry-run https://github.com/upfrontIO/microschema HEAD:ci-integration, if it works we use the URL https://github.com/upfrontIO/microschema. If it doesn't we convert the URL to https://<GH_TOKEN>@github.com/upfrontIO/microschema HEAD:ci-integration and run git push --dry-run https://<GH_TOKEN>@github.com/upfrontIO/microschema HEAD:ci-integration.
If it still doesn't work we return the The push permission to the Git repository is required. error.

github.com/upfrontIO/microschema is not a supported format and is an invalid URL as far as I know.

What do you get when you run in terminal the following:

• git push --dry-run https://<GH_TOKEN>@github.com/upfrontIO/microschema HEAD:ci-integration
• git push --dry-run https://github.com/upfrontIO/microschema HEAD:ci-integration

Can you also run semantic-release with the --debug option and post the logs here?

What do you get when you run in terminal the following:

Both work (but I have to enter the username and password for git push --dry-run https://github.com/upfrontIO/microschema HEAD:ci-integration as I haven't set any other environment variables in my terminal session).

To https://2bc307b681e4d4e8fe72825c53b525c3a4e3fa35@github.com/upfrontIO/microschema
   d6e1351..f99fb10  HEAD -> ci-integration

Debug Runs

With https://github.com/upfrontIO/microschema

[Semantic release]: Running semantic-release version 15.1.2
Username for 'https://github.com': livingdocs-automation
Password for 'https://livingdocs-automation@github.com':
  semantic-release:config options values: { branch: 'ci-integration',
  semantic-release:config   repositoryUrl: 'https://github.com/upfrontIO/microschema',
  semantic-release:config   tagFormat: 'v${version}',
  semantic-release:config   _: [],
  semantic-release:config   debug: true,
  semantic-release:config   d: true,
  semantic-release:config   'dry-run': true,
  semantic-release:config   dryRun: true,
  semantic-release:config   '$0': 'node_modules/.bin/semantic-release' } +0ms
[Semantic release]: Load plugin verifyConditions from @semantic-release/npm
[Semantic release]: Load plugin verifyConditions from @semantic-release/github
[Semantic release]: Load plugin analyzeCommits from @semantic-release/commit-analyzer
[Semantic release]: Load plugin generateNotes from @semantic-release/release-notes-generator
[Semantic release]: Load plugin prepare from @semantic-release/npm
[Semantic release]: Load plugin publish from @semantic-release/npm
[Semantic release]: Load plugin publish from @semantic-release/github
[Semantic release]: Load plugin success from @semantic-release/github
[Semantic release]: Load plugin fail from @semantic-release/github
  semantic-release:git Error: Command failed: git push --dry-run https://github.com/upfrontIO/microschema HEAD:ci-integration
  semantic-release:git remote: Invalid username or password.
  semantic-release:git fatal: Authentication failed for 'https://github.com/upfrontIO/microschema/'
  semantic-release:git
  semantic-release:git
  semantic-release:git     at makeError (/Users/Lukas/git/microschema/node_modules/semantic-release/node_modules/execa/index.js:172:9)
  semantic-release:git     at Promise.all.then.arr (/Users/Lukas/git/microschema/node_modules/semantic-release/node_modules/execa/index.js:277:16)
  semantic-release:git     at <anonymous>
  semantic-release:git     at process._tickCallback (internal/process/next_tick.js:188:7) +0ms
[Semantic release]: EGITNOPERMISSION The push permission to the Git repository is required.
semantic-release cannot push the version tag to the branch ci-integration on remote Git repository.

With //github.com/upfrontIO/microschema

[Semantic release]: Running semantic-release version 15.1.2
  semantic-release:git Error: Command failed: git push --dry-run //github.com/upfrontIO/microschema HEAD:ci-integration
  semantic-release:git fatal: '//github.com/upfrontIO/microschema' does not appear to be a git repository
  semantic-release:git fatal: Could not read from remote repository.
  semantic-release:git
  semantic-release:git Please make sure you have the correct access rights
  semantic-release:git and the repository exists.
  semantic-release:git
  semantic-release:git
  semantic-release:git     at makeError (/Users/Lukas/git/microschema/node_modules/semantic-release/node_modules/execa/index.js:172:9)
  semantic-release:git     at Promise.all.then.arr (/Users/Lukas/git/microschema/node_modules/semantic-release/node_modules/execa/index.js:277:16)
  semantic-release:git     at <anonymous>
  semantic-release:git     at process._tickCallback (internal/process/next_tick.js:188:7) +0ms
  semantic-release:config options values: { branch: 'ci-integration',
  semantic-release:config   repositoryUrl: 'https://[secure]@github.com/upfrontIO/microschema.git',
  semantic-release:config   tagFormat: 'v${version}',
  semantic-release:config   _: [],
  semantic-release:config   debug: true,
  semantic-release:config   d: true,
  semantic-release:config   'dry-run': true,
  semantic-release:config   dryRun: true,
  semantic-release:config   '$0': 'node_modules/.bin/semantic-release' } +0ms
[Semantic release]: Load plugin verifyConditions from @semantic-release/npm
[Semantic release]: Load plugin verifyConditions from @semantic-release/github
[Semantic release]: Load plugin analyzeCommits from @semantic-release/commit-analyzer
[Semantic release]: Load plugin generateNotes from @semantic-release/release-notes-generator
[Semantic release]: Load plugin prepare from @semantic-release/npm
[Semantic release]: Load plugin publish from @semantic-release/npm
[Semantic release]: Load plugin publish from @semantic-release/github
[Semantic release]: Load plugin success from @semantic-release/github
[Semantic release]: Load plugin fail from @semantic-release/github
[Semantic release]: Run automated release from branch ci-integration
[Semantic release]: Call plugin verify-conditions
[Semantic release]: Verify authentication for registry https://registry.npmjs.org/
[Semantic release]: Verify GitHub authentication

... the rest seems all honkey-dory

It seems like the dry runs are happening at different times in the two cases. And in the //github.com case the conversion to https://github.com/upfrontIO/microschema doesn't seem to work.

In the https: case it doesn't seem to try to push with the GH_TOKEN or it is not logged. To be fair the dry run with the GH_TOKEN is also not logged in the //github.com case. There it just works in the end.

So in the case of https://github.com/upfrontIO/microschema it seems the user/password you are typing in the prompt are wrong as shown by the error remote: Invalid username or password.
This error happens only when the password is wrong. If it was a permission issue you would get the error remote: Permission to upfrontIO/microschema.git denied to<username>.

With //github.com/upfrontIO/microschema the first attempt fails as it's a invalid URL. Although semantic-release manage to convert it to https://<GH_TOKEN>@github.com/upfrontIO/microschema.git and it works.

As far as I can tell everything works as expected:

• semantic-relrease test the permission with the URL provided, and if it fails and do not on CI you get the username/password prompt. If you enter a wrong password it fails
• If testing the original URL errors out semantic-release attempt with the GH_TOKEN as https basic auth

It seems your token is correct but the password you are entering is not.

I use a token as a password and it is the same I used for all the other tests. I didn't type it manually. The same token is also already in use in many other of our repos (although they use older versions of semantic release). And in all cases the GH_TOKEN is set as an environment variable. This token in GH_TOKEN is the same token I input manually if prompted. It's also the same token that worked in 15.0.3 an prior.

For me it doesn't make sense that it behaves differently depending on how the url is specified. Furthermore if the GH_TOKEN is provided I would expect semantic-release to use it.

But I don't mind much. I just observed this behaviour and thought you might find it interesting.

When you get the prompt it's a prompt from git not semantic-release. As far as I know you can't use the GitHub token as a password. You can use it as user and password with https basic auth.

If you run git push and you get the prompt for your username password and use the token as the password it would fail as well.

semantic-release behave differently because:

• In the first case you pass a correct URL, and your OS/git CLI is not set to save the password. semantic-release try to use the URL you provide first and as your not in a CI environment it lets git to ask you for your credentials. If you were in a CI environment it would directly try to convert the URL to use the GH_TOKEN
• In the second case you pass an invalid URL (as far as the git CLI is concerned) so the first try fails and semantic-release tries to format the URL with GH_TOKEN. This is a quite edgy case and semantic-release expect you to provide a valid git URL. And there is no good way for semantic-release to figure if the first call to git push --dry-run fails because the auth is wrong (and there fore try to add the GH_TOKEN as basic auth) of if it's because you provide a URL that is valid for node url.parse parser but not valid for git

The problem is that in the first case you try to use a GitHub token as a password which is not supported and in the second one use //github.com/upfrontIO/microschema as a Git URL which is not supported by the git command line.

I'm afraid there is no much we can do in semantic-release. Testing all the possible cases of valid URL for node url.parse that happens to not be valid for the git command line seems really out of scope.

Regarding github so there is no confusion, I'm talking about the personal access tokens: https://help.github.com/articles/creating-a-personal-access-token-for-the-command-line/

It seems I was mistaken and the token can be used in the git CLI as the password.

The prompt for username/password is from the git command line, not from semantic-release.
So it seems that for some reason the git CLI refuses your token as password and errors out with remote: Invalid username or password.
I don't know why, but it seems unrelated to semantic-release. If you do git push --dry-run https://github.com/upfrontIO/microschema HEAD:ci-integration you would have the same prompt and if you enter the same username/password you will have the same error.

I understand this is a weird bug. And it might only occur in some scenarios or locally. But at least in some indirect way (dependencies?) it is connected, because:

With a valid https://github.com/organization/repo url
In 15.0.2 when providing the GH_TOKEN it just worked. no prompting of anything.
In 15.0.3 when providing the GH_TOKEN it prompted for username/password and it worked.
In 15.1.2 when providing the GH_TOKEN it prompted for username/password and didn't work anymore.

By now, I'm just curious what is going on :)

And the token as a password thingy is super useful if you use two-factor authentication...

Do you currently have this behavior when you test the different version? As if you explicitly install semantic-release@15.0.2 and test?

Can it be due to something that changed in your environment?

Can it be due to something that changed in your environment?

I'm setting semantic-release up in a repo and I also removed the node_modules and reinstalled. Other than that I didn't change anything else today. At least on my machine it looks consistent. But at this point it probably makes sense if it can be reproduced by someone else (or just wait if others have similar issues).

I'm not sure I understand your test. Did you explicitly test with 15.0.2 by installing with npm install semantic-release@15.0.2?

I just saw than wen creating a personal access token it works if the token does not get any additional permissions.

If the token gets the repo permission it does not work anymore.

So it seems that depending on how you test if the connection works you get different results from the same token. As the permissions on the token are good enough in both cases (for a public repository that is).

So it seems that depending on how you test if the connection works you get different results from the same token. As the permissions on the token are be good enough in both cases (for a public repository that is).

I'm totally lost as that point. I still don't know if you are currently experiencing a different behavior when doing:

• rm -rf node_modules && npm install semantic-release@15.0.2 => Test
• rm -rf node_modules && npm install semantic-release => Test

Regarding the verification semantic-release verify the push access the same way in 15.0.2 and 15.1.2 with git push --dry-run https://github.com/upfrontIO/microschema HEAD:ci-integration so I don't understand what do you mean by "depending on how you test if the connection works you get different results from the same token".

The removal of the node modules doesn't change anything. It's consistent on my machine:

In 15.0.2 when providing the GH_TOKEN it just worked. no prompting of anything.
In 15.1.2 when providing the GH_TOKEN it prompted for username/password and didn't work anymore.

I tester the 3 version you mention and in all cases I get the prompt for the user password.
I set the repository url to https://wrong_user@github.com/semantic-release/semantic-release.git in order to force a prompt, otherwise git would use the user/password in my keychain.

Version 15.0.2:

npm install semantic-release@15.0.2
semantic-release --debug --repository-url https://wrong_user@github.com/semantic-release/semantic-release.git --dry-run

[Semantic release]: Running semantic-release version 15.0.2
Password for 'https://wrong_user@github.com': 

Version 15.0.3:

npm install semantic-release@15.0.3
semantic-release --debug --repository-url https://wrong_user@github.com/semantic-release/semantic-release.git --dry-run

[Semantic release]: Running semantic-release version 15.0.3
Password for 'https://wrong_user@github.com': 

Version 15.1.2:

npm install semantic-release
semantic-release --debug --repository-url https://wrong_user@github.com/semantic-release/semantic-release.git --dry-run

[Semantic release]: Running semantic-release version 15.1.2
Password for 'https://wrong_user@github.com': 

So as far as I can tell all 3 versions behave the same, at least on my machine.

Do you have a reproduction scenario that I can use to debug?

When passing --repository-url https://wrong_user@github.com/semantic-release/semantic-release.git --dry-run it also prompts for the user in all the cases you mentioned.

I guess the differences in the execution path depending on how the --repository-url is formatted cause the issues I see.

Maybe it is enough to reproduce the issue if you remove your user/password from the keychain (I myself don't have them in my keychain). I can't add the user/password to my keychain to test as I use two-factor authentication.

I tried remove everything git related in my keychain and remove the GitHub ssl key I have installed.
I still have the same behavior in all 3 versions, and everything work as expected when I enter my username and password.

The only change that was made between those version is that we expand the shorthand URLs (semantic-release/semantic-release => https://github.com/semantic-release/semantic-release/issues/708#issuecomment-374558363).

As far as I can tell semantic-release behave as expected. So I guess we can close this issue?

If you'd like to investigate why you have a different behavior between different versions feel free to do so, but you would have to troubleshoot and debug on your own as it seems it happens only with your environment. I can help but without detailed steps to reproduce I can't do much. If this investigation leads to finding an actual bug, then you can open a new issue with the relevant information.

Thanks for looking into it! And feel free to close.

As an input you may consider that if the GH_TOKEN is provided that semantic-release tries to push with the https url and the token first: https://<GH_TOKEN>@github.com/org/repo. In my opinion this would simplify testing locally if it tries environment variables that are used in the CI environment first. If semantic release tries with my username and password from the keychain first I can't really test locally if the token is correct. If it would be possible to pass the token as an argument to force a strategy that would also help.

In my usecase when testing locally before pushing the semantic-release setup to my repo the fallbacks are actually not really desirable.

Thanks again!

release 15.1.3 fixes the issues! 🥇

Now I can close it myself :)

That's the way if was working before (in 13.x.x or 12.x.x I think) but we changed that as in some case you want to have different method of authentication between the git CLI and the various API client. For example to use SSL keys for git CLI and GH_TOKEN only for the GitHub API.

There is many many uses cases around that, and they are really difficult to test as they involve proprietary Git repo, private GitHub/GitLab/Bitbucket instance, private CI, private authentication mechanism etc...

The implementation decision have mostly been made for running in a CI environment as it's the most common and the recommended way to use semantic-release.

Also we assume that in most cases when running locally authentication will not be necessary as the vast majority of users would have their credentials stored in some sort of keychain, or would have installed ssh keys, because you don't want to type your password every time you push...

15.1.3 fixes the issues!

I really don't understand how...But great news anyway!
For information 15.1.3 just prevent the git CLI to prompt for a username password when you are running on a CI. It doesn't change anything when you run locally (unless you use --no-ci).

We have an organisation setup with a bot-account that has the necessary permissions for CI integrations and publishing to npm. And we force all users to activate two-factor authentication. And I mostly want semantic-release to work exactly the same as it would on the CI for testing and as an emergency fallback when the CI is down but using the same tokens locally as we would on the CI.

But I completely understand if you optimise for the use cases the majority of your users care about.

Keep up the good work!

We just ran into the same issue. Hard one to debug :) Since we do heavy enforcement through js-gardener we'll update that there... blackflux/js-gardener#249

Hey I'm running into this issue on this repository: https://github.com/lastmjs/guesswork

I don't know what's wrong, I'm using CirclecCI, my token is updated and put in correctly, I'm using a version of semantic-release above 15.1.3, and my url starts with https://...any suggestions?