RFC: Proposal for new signing mechanism #37
Conversation
Co-authored-by: Roch Lefebvre <[email protected]>
Co-authored-by: Roch Lefebvre <[email protected]>
Co-authored-by: Roch Lefebvre <[email protected]>
|
On behalf of the Sigstore community, we'd like to voice our support here! Let us know how we can help make this a success. |
| 1. **Opt-out build-time signing of gems.** At this phase we will have gained experience and confidence with the new system. The RubyGems client will switch policy to make gem signing the default behavior, meaning that deliberate effort is needed to _not_ sign a gem. We expect this would create a rising tide of signed gems on rubygems.org as gem maintainers release new versions. Verification would happen automatically, but the lack of a signature would not be fatal. Gem authors would be able to set a policy that all their gem releases must be signed ([analogous to MFA configuration](https://guides.rubygems.org/mfa-requirement-opt-in/)). We would introduce a policy that the top most-downloaded gems must be signed ([analogous to the proposal for MFA](https://github.com/rubygems/rfcs/pull/36)). The `gem cert` command would be removed. | ||
| 1. **Opt-out install-time verification of gem signatures with strict policy.** At this phase signature verification is automated and gem signatures are the norm. We change verification policy to require signatures, treating the lack of a signature as an error. Users would be able to suppress this behavior on a gem-by-gem basis. This would provide further incentive for gem maintainers to sign their gems, and it would mean that the absence of a signature is a meaningful security signal. The server would warn older clients that unsigned gems will be subject to strict policy. | ||
|
|
||
| In Phase 1, signing will be opt-in, meaning that signatures will require `build --sign` and verification will require `build install --verify-signatures` (or equivalent `gem signatures` commands). But by Phase 3, signing has become default in `build` and verifying is default in `install`. This guide explanation will assume that we have reached Phase 3, as this is the most complete scenario. |
There was a problem hiding this comment.
| In Phase 1, signing will be opt-in, meaning that signatures will require `build --sign` and verification will require `build install --verify-signatures` (or equivalent `gem signatures` commands). But by Phase 3, signing has become default in `build` and verifying is default in `install`. This guide explanation will assume that we have reached Phase 3, as this is the most complete scenario. | |
| In Phase 1, signing will be opt-in, meaning that signatures will require `build --sign` and verification will require `gem install --verify-signatures` (or equivalent `gem signatures` commands). But by Phase 3, signing has become default in `build` and verifying is default in `install`. This guide explanation will assume that we have reached Phase 3, as this is the most complete scenario. |
| 1 gem installed | ||
| ``` | ||
|
|
||
| You may also use `gem install --verify-signatures` as in previous versions of RubyGems, but this is no longer necessary to trigger the verification process. |
There was a problem hiding this comment.
Does --verify-signatures refer to the old -P flag? If so, would it use the HighSecurity policy, or remove that option in favor of a single policy for everything?
There was a problem hiding this comment.
Our proposal doesn't leverage anything from the existing gem signing mechanism. -P and its possible values will eventually be discontinued, in our model.
In Phase 1, where the new system is opt-in, --verify-signatures is closest to MediumSecurity: not all gems are required to be signed, but the ones having a signature will be verified.
A gem is deemed to be signed if we can find at least one signature in Rekor using the digest of the downloaded file. Furthermore, at least one of those signatures needs to be associated to an email address listed in the file's gemspec.
Anyone can push a signature into Rekor for a given file. We only care about the people that also appear in the gem's own spec.
By Phase 2, we'd like to host a copy of the Rekor signature right in rubygems.org itself, alongside the gem. Less of a reliance on sigstore at installation time!
|
This is a really nicely written RFC, brilliant work Shopify team! The detailed breakdown of OIDC and OAuth2 flows in particular is really nice documentation I would love to see included in the sigstore documentation. I'm not a RubyGems contributor, just an interested observer, but there are a couple of points I was hoping the RFC would include:
|
We'll have some news to share on this soon! But we're hoping to get rid of that disclaimer very soon. |
I think yes, though we didn't cover it in this RFC (we left a lot on the cutting room floor). One critical function would be to email authors when their signature is published, to help them notice if something unexpected has happened. This is already done for gem pushes.
Yes, but we would gate Phase 1 on sigstore's reaching an official ™️ operational status. |
|
Great work! |
The answers I hoped for/expected, thanks @jchestershopify |
|
|
||
| <img width="60%" alt="Sign in to GitHub" src="https://user-images.githubusercontent.com/33674553/151442855-bc68dbcd-757d-40be-b38b-88d2be61e386.png"> | ||
|
|
||
| You then need to grant sigstore permission to request your email address from your provider account. This step only occurs on your first attempt to sign a gem using this provider. Only provider-verified email addresses may be used to sign gems. |
There was a problem hiding this comment.
Just to confirm my reading here: the OIDC provider verified email here is not being checked against one (or more) valid emails for the RubyGems user identity?
In other words: is there any formal relationship between the signing email (on sigstore's side) and the email(s) registered to a RubyGems account?
There was a problem hiding this comment.
In other words: is there any formal relationship between the signing email (on sigstore's side) and the email(s) registered to a RubyGems account?
In this design, no. We key instead on the email field in the gemspec, since that file is controlled at build time. Unfortunately there's no fixed relationship between email and the set of rubygems.org owners of a gem.
Where we think the connection back to owners will be possible is if we add push events to the transparency log. Put another way: authors would publish signature log entries, but rubygems.org would publish the push event log entries. In the latter case rubygems.org can annotate which owner made the push.
There was a problem hiding this comment.
Adding to the above, there's a security advantage in not tying signatures to the owner emails as verified by rubygems.org. It means that attackers need to achieve two account takeovers to publish a signed but malicious gem. One for a rubygems account, one for an identity provider account.
There was a problem hiding this comment.
Thanks for the clarification! That makes sense.
There was a problem hiding this comment.
Adding to the above, there's a security advantage in not tying signatures to the owner emails as verified by rubygems.org. It means that attackers need to achieve two account takeovers to publish a signed but malicious gem. One for a rubygems account, one for an identity provider account.
@jchestershopify I did not understand this point. I don't use gems so my understanding might be limited. Request you to please clarify.
Let us assume that an attacker has taken over the account of a gem owner. Now, is it not possible for the attacker to do the following:
- Modify the gemspec file to add a new
email(or replace existingemail) which is the attacker's email address - Sign using the newly added attacker
emailin the gemspec - Push the gem to rubygems.org using the owner's compromised account
Now, during gem install, the signature verification should work fine, since the gem is signed using an email associated with the gemspec.
There was a problem hiding this comment.
Adding to the above, there's a security advantage in not tying signatures to the owner emails
I don't believe so. If you have gem push rights, you can push a gemspec with any email addresses you want in it, of course. And therefore, I believe, easily add a new email address to be used for this proposed signing mechanism to the gemspec too. gem owner/gem push rights are still complete keys to the kingdom. (Am I missing something?)
Security mechanisms offer a mix of prevention, detection, and recovery from compromises. Gem signing would not prevent a compromised owner account from being used to push a gem. But it means that the attacker is easier to detect, because unless they also control the email address previously used, they need to introduce a new email address. This increases the chances of detecting an anomaly. Recovery is made easier because of the trusted public log. It’s not as good as prevention (for which we are championing MFA), but it’s an improvement nevertheless.
I think better integration between gem owner and the signature source of identity might actually be important for gem maintainer ergonomics and increasing ease of use and uptake here.
As you said, we should not create more ways for gem maintainers to screw up w.r.t. email addresses. Do you have any proposals for improving ergonomics, and to reduce the room for error?
There was a problem hiding this comment.
If gem authors are given extra work to authenticate their identity, then there would be the expectation that the authentication is used for authorization to make sure only gem owners are allowed to publish a package.
In this case, that means authorizing that the gem is signed by a gem owner, which means that the initial gem owner and changes to gem owners need to be signed and included in the transparency log such that they can be verified.
Currently, rubygems.org is the source of truth for the owners of a gem, so it needs to sign that initial gem owner list. Further changes should be signed by one of the gem owners. That way, the gem verification can check that a gem package is signed by one of the gem owners.
If the same were attempted to be done with fields on the gemspec, then we would need to verify against the list of owners in a previously verified version of the gem, not the gem being verified. It would also mean that a gem would need to be published to change the list of owners. In contrast, keeping gem ownership changes as separate events, that just need extra authentication, helps keep things working as ruby developers currently expect.
There was a problem hiding this comment.
If gem authors are given extra work to authenticate their identity, then there would be the expectation that the authentication is used for authorization to make sure only gem owners are allowed to publish a package.
We see signing ("I attest this build") and pushing as separate operations. In the original design signing is entirely independent of rubygems.org; in the alternative it gets tied in to rubygems.org identities. But it's still a logically distinct operation.
Currently, rubygems.org is the source of truth for the owners of a gem, so it needs to sign that initial gem owner list. Further changes should be signed by one of the gem owners. That way, the gem verification can check that a gem package is signed by one of the gem owners.
The idea of having existing owners sign new owners is an interesting one. I think we can reuse that when we revisit the topic of further log entry types in future.
There was a problem hiding this comment.
Signing and pushing are distinct actions, but the ability to do both should still be needed to publish a gem that will be accepted by a rubygems client for installation, such that they aren't really logically distinct. If you mean that the rubygems.org shouldn't need reject the publishing of a gem without a valid signature, then that's correct that it shouldn't be needed for security reasons (it would just keep invalid data out of its database).
In the alternative where rubygems.org verifies the identity, then it should be possible to make the gem signing seamless from the perspective of the gem author, since they would otherwise have their identity verified twice in a row (for signing then pushing). Since there is no security benefit from authenticating twice when these are done together, there should not be any extra work. The seemless experience also means that there wouldn't be a need to opt-in to adopt this feature. The only difference would be that the rubygems authentication for publishing would get persisted as a signature in the gem and events in the transparency log for verification.
There was a problem hiding this comment.
I agree that signing and pushing are distinct and that in the long run both should be verified (we touched on it in "Other kinds of log entries"). But we considered it out of scope for this RFC. We're trying to boil the tea kettle first.
|
Ditto all the praise above: this is a very exciting improvement to the Gem ecosystem! I'm currently working with Google and the PyPA (the team than runs PyPI) to implement a nearly identical (and This is outside of the scope of the scope of the RFC as-is, but something for the RubyGems ecosystem to consider: PyPI is also gaining support for credential-free authentication via trusted OIDC providers like GitHub (and eventually others) -- we plan to allow PyPI users to mark a particular repository and CI workflow as "trusted," allowing us to derive a temporary API key from the action's OIDC token. A user can then GitHub Actions to publish their package without having to configure an API key, add it as a secret, etc. Something to consider for a future capability within RubyGems as well! |
| [rest of issuing (root) certificate] | ||
| -----END CERTIFICATE----- | ||
| ``` | ||
|
|
There was a problem hiding this comment.
Fulcio will also return an SCT (https://datatracker.ietf.org/doc/html/rfc6962#section-3), saying that Fulcio has uploaded the certificate to its own transparency log (accessible at ctfe.sigstore.dev, verified here in cosign).
After getting the certificate from Fulcio, do you want to verify the SCT? Note this is a different verification key than the Rekor key.
There was a problem hiding this comment.
Very cool. I was aware that Fulcio's cert goes into a CTL, but I didn't know the details. Here are the headers coming from a POST to sigstore's signingCert endpoint
{
"date": "Mon, 31 Jan 2022 14:04:35 GMT",
"content-type": "application/pem-certificate-chain",
"content-length": "2006",
"connection": "keep-alive",
"sct": "eyJzY3RfdmVyc2lvbiI6MCwiaWQiOiJDR0NTOENoUy8yaEYwZEZySjRTY1JXY1lyQlk5d3pqU2JlYThJZ1kyYjNJPSIsInRpbWVzdGFtcCI6MTY0MzYzNzg3NTY3NywiZXh0ZW5zaW9ucyI6IiIsInNpZ25hdHVyZSI6IkJBTUFTREJHQWlFQTBQNkZmVnYzSUhJd1JqSXJJcUR0OXNZN1d0WTY2S2dLdHJJNjl1QjVRdzBDSVFETUg4ZkF0ODBVRGp2c042U2NvVDV2NGhCT3FuSWhtZlN5eFRtbFhiSHJQUT09In0=",
"vary": "Origin",
"strict-transport-security": "max-age=15724800; includeSubDomains"
}After getting the certificate from Fulcio, do you want to verify the SCT?
Pending the resolution of how we'll manage the various sigstore verification keys you mentioned in another comment, I believe we should check the signing cert's SCT. Same goes for checking the Rekor log entry's SCT.
| "inclusionProof": { | ||
| //… | ||
| }, | ||
| "signedEntryTimestamp": "MEUCIQDWwF3ehGj+9BBsVdnm5Bg[...]eooKBPZkXLLDVl9t72FkY/VUHw=" |
There was a problem hiding this comment.
Do you also plan to verify the Rekor entry? Note that you'll need to distribute the verification keys securely too. See comment below about TUF.
There was a problem hiding this comment.
Yes, I think the log entry should be verified, whether it's retrieved from Rekor itself or whether it's cached on rubygems.org. I don't yet know how the gem client would acquire/manage the key needed to check the signedEntryTimestamp.
As you throroughly explained in your TUF comment below, this is not trivial.
There was a problem hiding this comment.
I wouldn't suggest pulling it directly from Rekor on each verification, since you'll be trusting the log to not lie or be compromised. You could cache the key at a trusted source like rubygems.org. You'll need to figure out an update mechanism too, which minimally could be supporting multiple verification keys.
I think it's also important to verify the Fulcio certificates without fetching the root from CA issuers. The AIA information for CA issuers is meant for chain building. Also, since Rekor and Fulcio are separate systems, I wouldn't necessarily trust that the certificate put into Rekor chains up to a trusted root.
| Authority Info Access: CA Issuers - URI:http://privateca-content-603fe7e7-0000-2227-bf75-f4f5e80d2954.storage.googleapis.com/ ca36a1e96242b9fcb146/ca.crt | ||
| ``` | ||
|
|
||
| An issuing certificate does not change after creation. It may therefore be cached on the gem client. Fulcio's root certificate will infrequently change. |
There was a problem hiding this comment.
How will you trust the Fulcio root certificate and Rekor entry? It's important to either pin or securely distribute the verification keys and certificates, or verify that they chain up to Sigstore's TUF root. If you want to verify a Fulcio signed certificate, the Rekor entry, and the SCT from Fulcio, you'll need a mechanism to securely ship the Fulcio cert, Rekor public key, and Fulcio CT log public key respectively.
One option is TOFU (trust on first use) - The client should persist the above three verification keys/cert on first verification. However, they will be rotated over time. You will then need to inform users that the values have changed. You'll end up creating something like SSH, where users see a warning that keys have changed, and users end up becoming complacent and simply clicking through without verifying the changed value.
A preferred option is to verify these keys/cert using Sigstore's TUF root. You mentioned TUF as an alternative to signing below, but for Sigstore, TUF is complementary to signing artifacts, as it's used to verify the verification keys. Cosign ships a trusted TUF root. Changes to the TUF root and its signed targets can be verified and trusted, since all changes will always chain up to the bundled TUF root.
Another alternative is bundling the verification keys/cert with the client, but once again, you'll need to figure out a secure update mechanism.
With your design, you must trust that the Rekor log is not compromised, since you use that to fetch the signed certificate. By verifying the Rekor log and verifying the its public key, you mitigate the attack where the Rekor log is compromised.
There was a problem hiding this comment.
In case it's not clear, we're talking about getting trustworthy keys to verify the signedEntryTimestamp on each rekor log entry, and also the Fulcio certificate signatures.
After brushing up on my understanding of TUF, I have a better grasp of your questions and proposals.
IMO, trust on first use might work for gem clients in a manual setting (e.g. one's own stable development machine), but it's not so trustworthy if the gem client is continuously reinstalled as part of some headless/CICD environment. Every reinstall starts with a leap of faith that the online keys are currently trustworthy. This could be exploited.
I'm leaning towards shipping with a trusted root, and following the TUF client workflow to acquire/verify the latest lineup of metadata files (newest root, timestamp, snapshot and target files) from sigstore. We then download the keys from sigstore as well. Implementing thre client flow is not trivial, but there are precedents to follow. Perhaps this would make most sense as a self-contained tuf-client gem?
As a gem client release gets older, it will need to perform more interations of the update root role bit in order to reach a fresh state. Newer major/minor/patch releases of rubygems ship with a fresher trusted root.
I'm not too clear on whether https://github.com/sigstore/root-signing is meant to be the TUF metadata & key repository, or simply a tool to generate them? I see a lineup of keys here, but the accompanying snapshot & timestamp metadata files have expired.
There was a problem hiding this comment.
I'm not too clear on whether https://github.com/sigstore/root-signing is meant to be the TUF metadata & key repository, or simply a tool to generate them? I see a lineup of keys here, but the accompanying snapshot & timestamp metadata files have expired.
Indeed, this provides a root for the whole log using a TUF root. It would be possible to e.g., provide a delegation to rubygems and use that TUF delegation as a "root" for rubygems packages that is based off the sigstore root. It would be understandable if that technically outsources the root to a third-party project.
I guess what I'm trying to say is that that repository also holds the tools to help you create your own root in the same way that sigstore does, so I think it's up to the community to decide if the root should be detached from sigstore, or a delegation from it 👍
There was a problem hiding this comment.
@rochlefebvre this sounds similar to the TUF + sigstore proposal in this blog post. That would let you use a gem-specific TUF root with sigstore.
There was a problem hiding this comment.
If you don't have other trusted metadata you need to verify and update, then I'd lean towards bundling the Sigstore TUF root. Using the client workflow, the Rubygems' client would always fetch the latest TUF metadata. +1 to what @SantiagoTorres mentioned, if you have additional metadata you want to sign in the future, you can choose between a delegated role chained up to the Sigstore root, or maintain a second Rubygems TUF root.
As a gem client release gets older, it will need to perform more interations of the update root role bit in order to reach a fresh state. Newer major/minor/patch releases of rubygems ship with a fresher trusted root.
Something to note: Roots can be cached locally, so the update mechanism doesn't make many remote calls.
There was a problem hiding this comment.
I am not clear on how a gem client would know which Rekor public key to use to verify a particular signed entry timestamp. Don't the keys change over time? Which target is the intended key? Are there key ids somewhere?
I have a similar question for which key to use to verify the code signing and issuing certificate signatures. In our current proposal, we're just trusting the issuing certificate's embedded key, which we find using AIA.
There was a problem hiding this comment.
I have part of an answer for my previous questions, thanks to the info in sigstore/cosign#1273 (comment)
sigstore's TUF targets are not yet versioned (as of v2). Starting in v3, and thereafter, the latest targets metadata will list all supported certificate and key files: current, expired, rotated, but not revoked.
For example, a TUF client can follow the update workflow to download current and expired Fulcio root certificates. All of these certs may be trusted, as long as they were valid at the time the gem was signed. A signing certificate identifies its parent certificate via the Authority Key Identifier claim.
Past and present Rekor public keys may also be downloaded using the targets metadata. It's not clear to me which key file we should use to validate the log entry signature. Trying all of them is one way to do it... 🤷
There was a problem hiding this comment.
Sorry I didn't respond to the first comment!
Before sigstore/cosign#1273 (comment), there was no way to verify entries with older keys. The expectation would be that you could only verify entries signed with the latest set of keys/certs from TUF. With this new design, we'll store all the current and previous versions of the key in TUF metadata.
To answer your question about how to verify a Rekor signature with multiple keys, yep, right now we're just trying all of them - code
For verifying the code signing certificate, the approach is more elegant - code. We use a CertPool to load in all trusted certificates. Verify will find all valid chains starting from the leaf certificate, through intermediates, down to the root, using Authority Key Identifier to find the correct chain. Hopefully there's an equivalent Ruby library like https://pkg.go.dev/crypto/x509.
For verifying the Fulcio CT log key, we do the same verification logic as with the Rekor key, trying to verify against all trusted keys - code
As for your earlier comment about AIA, I'd recommend avoiding using AIA to pull down a certificate. AIA isn't meant to find a root of trust, it's meant to be used by clients for chain building. If you use the TUF metadata, this should contain the Fulcio root CA certificates. When you implement storing the signature alongside the gem, i'd recommend also including the certificate chain (from leaf to root, in case we add subordinate CAs at some point).
There was a problem hiding this comment.
One other thing to note: the LogID field within the Rekor signed entry timestamp (used for offline verification) is the SHA256 hash of DER-encoded public key, which marginally helps in finding the right one.
There was a problem hiding this comment.
Before sigstore/cosign#1273 (comment), there was no way to verify entries with older keys. The expectation would be that you could only verify entries signed with the latest set of keys/certs from TUF. With this new design, we'll store all the current and previous versions of the key in TUF metadata.
@asraa and I talked about this last week: it is definitely possible with TUF, it just depends on what targets (i.e., keys/certs) you sign. I don't remember the entire context now, but perhaps Asra or Hayden can add more context.
|
Thanks for this RFC 💪 🙏. Looking at the sigstore login picture, does it mean you need GitHub, Google or Microsoft account to be able to sign gem? |
|
|
||
| ### Alternative flows | ||
|
|
||
| The design outline above requires interaction with a browser. In some scenarios this will be undesirable or unworkable. We believe that alternative flows are technically feasible, but we have not prototyped such flows. Ideally such a design would be settled by the start of Phase 2. As this work develops, we expect to open a follow up RFC. |
There was a problem hiding this comment.
We should roll non-interactive in initial "production grade release" of this feature. There are "top" gems released automatically (like Amazon Web Services one) and this should not prevent signing them IMHO.
There was a problem hiding this comment.
I think this is answered by #37 (comment).
There was a problem hiding this comment.
As part of our work, we have developed a proof of concept system that works as a gem plugin.
🤔 Looking at the current (reference) implementation, I see a huge list of big dependencies (like
activesupport). If we plan to make this part of RubyGems by default, we will need to find way how to incorporate this feature with minimal dependencies, since RubyGems itself (by design) can't have any dependencies on gems....
I'm usingactivesupportas an example only here. Same can be applied to any dependency.🤔 On the other side (as also mentioned in RFC) OAuth2 is super complex and implementing it again just for this purpose doesn't make any sense.
We leveraged many dependencies to quickly develop our proof of concept, but we know that this is incompatible with how rubygems is designed. We're prepared to do the detailed work to put it on a serious gem diet. We didn't want to surprise the community with production code before building consensus through the RFC process.
There are many gems, like activesupport or faraday, which are definitely optional. Then there's libraries that facilitate OAuth, OpenID Connect, JSON Web Token, JSON Web keys, and even browser interactions. To a certain extent, any one of those libraries could be vendored or reimplemented, on a case-by-case basis.
OAuth and OpenID are huge specs, with many supported flows, and so are the gems which support them. In our reference-level explanation, we can see that the authentication bit isn't so bad. RubyGems doesn't need to go full OAuth2/OpenID Connect compliant to interact with sigstore. TBH, I have not delved into JWT and JWK yet, so I don't know how heavy of a lift reimplementation would be.
Opening a browser or tab on many supported platforms is non-trivial. The Launchy gem we use takes care of many details. I'd consider vendoring it to solve an otherwise hairy problem. It only depends on the addressable gem at runtime, so there's that!
My initial idea is to keep this as a gem plugin, support
--signparameter by default (without having gem installed) and install this gem if this parameter is used and the gem plugin is missing (similar what we do currently for missing bundler version). That way it can still use the dependencies and make it usable in default RubyGems installation. As a side-benefit we can ensure latest supported version is installed.
This is a very clever way of installing a plugin on demand. It could work in the early parts of Phase 1, while signing is still opt-in. However, we'd like gem signing to eventually become opt-out. That would make the plugin required for gem maintainers. Having a support matrix of m RubyGems versions times n plugin versions is undesireable, especially if it's security sensitive.
There was a problem hiding this comment.
However, we'd like gem signing to eventually become opt-out. That would make the plugin required for gem maintainers. Having a support matrix of m RubyGems versions times n plugin versions is undesireable, especially if it's security sensitive.
I don't see any problem having this approach used also with opt-out signing. It will just install the dependency when it will be missing the same way as it will do it during opt-in phase. We can hardcode the gem version used on RubyGems release (for example following RubyGems versioning). Gem would be required, yes, but it will be auto-installed when missing on first usage. See example (verbose) output.
# first run
$ gem build my.gemspec --sign
sigstore rubygems plugin not installed, installing now
# following run
gem build my.gemspec --signLater in phase 2:
# first run
$ gem build my.gemspec
sigstore rubygems plugin not installed, installing now
# following run
gem build my.gemspecThere was a problem hiding this comment.
I'm really warming up to this idea. The proposed gem signature verification flow could still be implemented in the gem client directly, since it's fairly straightforward.
Gem signing is where things get complicated, functionality-wise. If we can leverage dependencies to do it, then great! Install on first use makes sense, as the vast majority of RubyGems users don't build gems.
I feel this would be a first party plugin, and would have its own repo under the rubygems GH organization. Is this your expectation as well?
There was a problem hiding this comment.
I feel this would be a first party plugin, and would have its own repo under the rubygems GH organization. Is this your expectation as well?
Yes!
Excellent. My colleague @jchestershopify is suggesting we organize a cross-ecosystem working group on sigstore Slack. Stay tuned!
We expect to support CI OIDC tokens assigned to jobs/actions as an alternative to performing a web sign in. Fulcio already has support for GitHub action tokens. I don't have the best grasp on this scenario, but I hope others can propose solutions. In the meantime, sigstore auth already supports single use refresh tokens, which is an attractive way to decouple web authentication from gem signing. Here's what we're thinking:
$ gem signatures --token
<same browser-based sign in as shown in RFC>
Signing token: ChlqaWVxc2RobjVkNTczankzY3Y0bjdoYTd1EhlyMzUzeWtjM3NnNHBrcHd2bmt0Mmxyd2k0Later on, in some job or as part of a script, this token can be used instead of having to interact with a browser: $ gem build --sign --signing-token ChlqaWVxc2RobjVkNTczankzY3Y0bjdoYTd1EhlyMzUzeWtjM3NnNHBrcHd2bmt0Mmxyd2k0 foo
Gem `foo` signed as [email protected]Refresh token are much longer-lived than the ID tokens we get from the regular flow (which expires after one minute). |
For right now, yes. As sigstore adds more supported identity providers the list will grow. |
🤔 Looking at the current (reference) implementation, I see a huge list of big dependencies (like There are few gem exceptions being vendored under RubyGems namespace. https://github.com/rubygems/rubygems/blob/975f9dcbef882bf457dab200d31bc87044e9a2b0/Rakefile#L88-L125 But vendoring whole 🤔 On the other side (as also mentioned in RFC) OAuth2 is super complex and implementing it again just for this purpose doesn't make any sense. My initial idea is to keep this as a gem plugin, support |
Are there any plans to support kind of "custom" account using just an email? That way no "vendoring" is going to happen. |
Not as such, given the way the flow works. Part of the security guarantee comes from having a third party vouch that the signer legitimately controls the given email address. The list isn't fixed; if someone sets up a non-vendor identity provider that's trustworthy and reputable enough, it could be added to the list. |
And is there any reason to not be able just to provide your email on your own instead of relying on (for example) GitHub handing over your email to the service? If I understand it well, it is all about getting the email. |
The difficulty is that this puts us back where we are: you have to manually decide to trust a claim, rather than falling back on independent roots of trust. Right now I can upload a gem with The approach we outline forces you to prove, with a third party attestation, that you really do control the email address. |
I'm not sure I do follow, but what would be different when I provide email and just confirm in an received email I have access to it? I can understand if sigstore is in beta and early stage,and this is not implemented since going OAuth to get email could be easier. But I don't see any blocker to add this "manual" option as well. I can check (open discussion) somewhere on sigstore side. |
I think I see your point now. What you're proposing would be to provide a rubygems-operated OIDC identity provider. Which would be possible, but there are two serious drawbacks.
|
I understand OAuth2 and OIDC is complex, but I don't understand why we need this complexity just to get an email address. Being able to login using RubyGems credentials would be nice, but I still think just providing email (with verification email link) would be the simplest way in here. If we stick with current options, we will "vendorize" whole feature since it will require Google, Microsoft, or GitHub account to use. It actually adds security to open source software traded for requirement of having 3rd party account on proprietary service. That doesn't make any sense IMHO. Me, personally, would not use such feature. So I'm looking for alternatives to make it useful without any 3rd party service being involved as well. 🙏 |
|
I understand your concern and it's valid. We think this problem will be faced by everybody who is currently working on sigstore-based systems (we are aware of efforts by NPM, Gradle and PyPI). Can we have a little time to huddle on this question? We'll try to work out what can be done. |
|
@simi has a point. During my work with and in RubyGems, I've encountered several gems developers/maintainers including core Ruby members that do not have their code in Github and use private domain-based emails. While I have a Github account and I would be ok using it, I spent 3 years moving away from gmail and I can imagine people that do not have any of the ones mentioned above. That said, I am in favour of improving the current signing capabilities of RubyGems as I do not see a lot of usage of it. |
|
As discussed above, maintenance of an IDP is not trivial. This will require significant knowledge of OIDC/OAuth, along with the operational costs and security risks associated with running an IDP. I would recommend focusing on implementing signing support initially using public IDPs, and revisiting running a RubyGems IDP during a later iteration. For those who are not comfortable with including their email address in the issued certificate, it's worth noting that Sigstore supports Github Action OIDC identities, as noted in #37 (comment). Signing that is initiated using a Github Action will only include information about the repository in the issued certificate. Github Actions is also beneficial for those who want automation, without a browser.
Adding onto #37 (comment), we will likely need to make a few changes to Fulcio to support whatever is decided. The subject of the token could either be the profile URI as a URI Subject Alt Name or the username of the Gem owner (Fulcio could append a configured domain to construct a SAN URI, such as |
|
|
||
| We are less concerned about unavailability risks at verification time. This is because we believe it will be possible to cache Rekor log entries into the RubyGems bucket. Essentially, a file containing the signature and signing certificate for a .gem would live side-by-side with the .gem. Clients could retrieve the signature and certificate from the RubyGems bucket, without needing to connect to sigstore (though retaining the option to do so if sufficiently paranoid). Since installation operations greatly outnumber builds, the impact of any sigstore service problems would be contained to gem authors. | ||
|
|
||
| ## Rationale and Alternatives |
There was a problem hiding this comment.
Was any thought given to the relatively new ssh-keygen signing functionality? Git recently added support for signing git commits with SSH keys. We could do something similar with a new "gem" namespace for signing.
I see a few advantages to this approach over sigstore:
- Many developers will already have an SSH key used for various purposes, including access to GitHub via
git push/git pull, access to remote servers, etc. - It removes the need for a fourth party beyond the gem maintainer, the gem consumer, and RubyGems to be involved in the signing and verification process
A difficulty with ssh-keygen's signing is that it requires each gem installer to maintain an allowed_signers file with each maintainer's email, but that could be partially alleviated by RubyGems running a certificate authority signing keys as they're used to sign gems. We could explore automatically adding that CA to ~/.ssh/allowed_signers, coaching users to add it, etc.
There's also an issue of maintaining the SSH keys (at least the public), which tend to be considered disposable. I do think there would be more coaching needed to have maintainers preserve their SSH keys, but as only the public key is required for verifying we might get away with just RubyGems tracking those.
There was a problem hiding this comment.
Hey!
Sigstore maintainer here. I don't really ssh keys as orthogonal to the sigstore based ones. We can definitely add support for them in cosign, and they're already supported in rekor actually. The main issue is really the part you outlined:
There's also an issue of maintaining the SSH keys (at least the public), which tend to be considered disposable
Rekor/Fulcio basically just automate the dispensing of disposable keys to help with this problem. We could do ssh ones or anything. We can also support people that do like to manage their own keys with the same systems.
There was a problem hiding this comment.
What if rubygems.org simply let gem owners register public keys authorized for signing? Ala github. (And let you revoke them etc, so I guess it woudl keep a record of keys that were authorized for signing during certain dates, for verifying historical gems). That is much easier infrastructure to build than an "OIDC/OAuth IDP", I think?
There was a problem hiding this comment.
That was kind of my thought, and the CA allows them to make allowed signers maintenance easier.
There was a problem hiding this comment.
What if rubygems.org simply let gem owners register public keys authorized for signing?
SSH signing keys are frequently lost. If users lose track of the keys, they'll need to revoke them. We then can't trust the artifacts that were signed with that SSH key. Ephemeral keys where you trust an entity that signs the keys are going to be resistant to this.
There's very little support for SSH CAs out there. I know that Smallstep offers an SSH CA. In my opinion, maintenance of a CA is going to be the same as maintenance of an OIDC IDP, with respect to key/cert rotation and the necessary security posture.
I also worry this would impact automation. For example, Github only issues OIDC tokens. There isn't wide library/platform support for SSH signing, we'd be relying on a very new feature.
There was a problem hiding this comment.
SSH CAs do not have a CRL function as regular X509 Certificates, making revocation a more manual process of having to add the revoked keys to a file on each client. This makes using SSH Keys as a code signing entity harder to implement (to mitigate the lack of revocation at Keytos we create short term SSH certificates but that would not work for code signing since the packages would have to be re-signed with a new key fairly frequently. If going the certificate route I would recommend doing X509 Certificates as these CAs do have the ability to easily revoke certificates.
There was a problem hiding this comment.
Briefly, an SSH scheme looks almost exactly like the current x509-based scheme. It would face the drawbacks we outlined in "Motivation".
|
Could you include both guide level documentation and technical documentation for a revocation model? Many supply chain failures with libraries involve unchecked patches or developer machine exploits that produce or enable signed package productions that would be best later revoked. What will that flow look like? |
Good question! We briefly and indirectly touch on that in the unanswered questions. We envision that the immutable transparency log would contain more than gem maintainer signatures. 3rd party reviews, gem yanks, changes to a package's owners, etc. A These sorts of attestations and gem installation policies build on top of what we are proposing in this first RFC. They are currently on the cutting floor so we can pitch a more concise approach to gem signing. |
|
Here are some early thoughts and clarifications on Jacques' Do everything with Rubygems.org IdP scenario. I'll state the obvious here: these are relatively new ideas, subject to change. Rubygems.org is the only authority on its profiles. If we intend to have "https://rubygems.org/profiles/flavorjones" as the signing subject instead of [email protected], then sigstore needs to receive a bulletproof attestation that the requestor can indeed access that rubygems account. In the RFC's detailed signing flow, steps 1-10 show the sigstore's instance of Dex Requirements are TBD as of this writing. I can think of a few:
At the risk of solutioning too early, I looked at doorkeeper, a gem that adds OAuth support for Rails apps. It's mature, supports many standards, has a lot of contributors, and a permissive license. Here's a rubygems.org PR that takes it out for a spin. I believe that doorkeeper experts and sigstore/Dex experts could hash out a minimal configuration that satisfies sigstore requirements. |
|
Update on using a rubygems.org profile URI as the signing subject. As a precondition for supporting a new identity provider, the sigstore maintainers require that the subject identifier (e.g. the profile URI) must not be reassignable to another entity. If https://rubygems.org/profiles/qrush represents User 1 today, it cannot resolve to User 2 in the future. This is incompatible with our preliminary ideas for using a profile URI having a user handle in the path, because user handles are not permanent. I can think of two ways to reassign a user handle to someone else:
Handles are case-sensitive. Foo and foo are different users. Profiles may also be referenced by their user record ids. https://rubygems.org/profiles/qrush is also known as https://rubygems.org/profiles/1. The profiles api endpoint also allows either addressing scheme, and returns both: id-based profile URIs are stable, and may not be reassigned nor reissued as part of another user creation. If the profile is deleted, then the URI points to nothing. This too could change if the repo supported some kind of user soft deletion (or just copied the deleted user into a For these reasons, I recommend that we use the user id version of the profile URI as the signing certificate's subject alternative name. The handle could also be recorded in a custom extension, with the understanding that it is not authoritative beyond the time of signing. |
|
Hi @simi! Early on in the RFC review process, you expressed interest in finding alternatives to relying on 3rd party identity providers as the primary way to identify gem signers. You entertained the idea of having rubygems.org as an identity provider. I'm all for exploring this avenue; it avoids a few problems from the 3rd party IdP/email approach. I have done some preliminary research, and I have started figuring out what this approach entails for both rubygems.org and for sigstore. Before people put too much time researching the option, I'd like your take on the intended result. What do you think of this alternate proposal? |
|
A few weeks ago, Jacques proposed an alternative approach to signing gems using an identity provider-verified email address. Instead, gems would be signed using a maintainer's RubyGems profile. As a reminder, here are some pros and cons to this approach:
To me, the killer feature is the clear identity of the signer in the signing certificate: the rubygems profile itself. That got me thinking about other public identities that gem maintainers may wish to sign their work as. Maybe a GitHub or Twitter profile? A GitHub profile-based signature makes sense to me. It has the same pros as an RubyGems IdP, minus the vendorizing. It also sidesteps most of the cons. I don't have any great ideas on how RubyGems profiles and GitHub profiles may be linked: storing the GitHub user id, perhaps? To be clear, I still think the RubyGems IdP option needs to happen, perhaps as the default signing provider. If sigstore starts supporting profile-based signing, then work could begin on the various other parts of Phase 1 (i.e. opt-in) gem signing sooner. The RubyGems IdP work could start later, or progress in parallel. |
The con here, of course, being tied to a single commercial provider. |
GitHub profile signatures cannot be the only option, no. Having a GitHub account should not be a prerequisite for being a gem maintainer. That's why RubyGems profile signatures are still on the table. The latter take more effort to implement, and are trivial to forge in the event of a rubygems.org account takeover. I'm suggesting that we look at supporting GitHub profiles first. That gets the ball rolling, but isn't sufficient for enabling opt-out gem signing (what the RFC refers to as Phase 2). We'd need a RubyGems IdP for that. sigstore is the gatekeeper for supported identity providers. Really, any type of public profile that is meaningful to the gem ecosystem could work: GitLab, Twitter, etc. |
|
Hi everyone! Lovely to see you all in here thinking about the future of rubygems security! I just wanted to chime in and say that the rubygems IdP solution requires setting up federation with sigstore, which strangely enough is something that I did a few months back for a side project (hi @dlorenc :D ). So I've already got a pretty good idea of how to code it up and it's not too difficult. The sigstore team has been great at helping get things up and going. On the topic of encoding of ownership, I'd recommend that rubygems implement it via the |
|
Oh, I didn't introduce myself. I'm Evan Phoenix, long time Ruby Central board member and long time rubygems.org admin. Currently the hosting bills for rubygems.org fall to me and thus a modicum of responsibility. |
|
Sorry, maybe I missed something, but has this RFC discussed how to handle developers who don't wish to sign their gems for whatever reason? I understand that there are phases in the RFC, but I can't imagine a world where everyone signs their own gems unless it's 100% mandated/required, and that can be difficult to achieve. Have you considered the option where RubyGems signs for gems by default unless developers provide their own sigs? (Similar to the PEP 480 model.) Again, apologies if I missed something somewhere. |
Per my understanding of https://github.com/Shopify/rfcs/blob/new-signing-mechanism/text/0000-introduce-a-new-signing-mechanism.md#guide-level-explanation, there is opt-out phase. You should be able to push the gem without signing using kind of CLI argument, but it will be enabled by default. |
Correct, it would be an opt-out scheme. There would be an escape hatch but it would be as narrow and uncomfortable as we can make it. But eventually owners of most-downloaded gems should be required to sign, just as we're proposing that owners of most-downloaded gems should be required to enable MFA. |
We foreshadowed that other kinds of log entries should be introduced in future, including a push / publish entry. In my view this is functionally equivalent to the repository signing (just as gem owner signing can be seen as functionally equivalent to a build attestation). It's an example of where I would like different ecosystems to agree on a schema in the long run, but otherwise out of scope for this RFC. |
Could you walk through an example? I've never felt like I grokked SPIFFE/SPIRE. |
Hmm, interesting. What's not clear to me right now is how client-side verification would work, especially with recursive dependencies. If I'm a package manager, do I turn on verification for all or no packages? How do I know which ones are supposed to be signed?
Interesting idea about other kinds of log entries. However, I'm not sure that they are functionally equivalent to repository signing. As I mentioned, it's not clear for package managers which gems should be signed by developers vs repositories. My advisor Justin Cappos and I looked at this issue a few years ago. We were very interested in finding strategies that would help effectively build the number of packages signed over time. In particular, you may be interested in Section 7 of our paper, where we used a month of downloads from PyPI to evaluate questions such as:
And so on and so forth. Without going into the specific strategies, the figure above shows how we can reduce the body count, so to speak, as we adopt different strategies over time. Sorry, I hope this wasn't a digression, but I wanted to get my thoughts out while they were fresh in my head. I hope this was helpful. Please let me know if you have questions. My main point is that I wanted to know: (1) how we convince projects to sign over time, and (2) how package managers can reliably know which projects should be signed by repositories vs projects/developers themselves. |
Co-authored-by: Trishank Karthik Kuppusamy <[email protected]>
|
As part of the Securing Software Repos working group's efforts, @znewman01 published a short paper on different options for software repos in adopting sigstore. The paper canvasses a lot of what we've discussed in this thread so far, including whether to have RubyGems act as the IdP or continue to delegate it to others (including a possible non-vendor "neutral" IdP). |
|
I think the overall idea of sigstore and not having long-lived keys is a good one. One thought on the issue of a single point of failure mentioned in the pros/cons summary above:
If we keep the email or a RubyGems account as the primary identity/proof [signing subject], but also add support for secondary proofs. These secondary proofs could be long-lived private keys stored in the users device or e.g. a Yubikey (old school private/public keys), or secondary attestations from another individual (via the proposed IdP flow, but this user wouldn't be the primary proof). Basically, these secondary proofs would be allowed to change, but them doing so would be a useful signal. For example, if an author loses their private key for a secondary proof (new computer, etc) the gem client could show a warning. Popular gems could always have a another individual providing secondary proof, and if that person changes a warning would be shown. It would be similar to the safety numbers used by Signal, or the device identifier that most SSO systems will store on a device (to keep track of device changes, and often ask for additional proof). So, if someone have their account compromised at RubyGems, the attacker would have difficulty compromising the secondary proof (Yubikey or another persons account) at the same time. So a published gem update would draw more scrutiny if the secondary proof was missing or changed. A legit owner replacing their secondary proof (computer or Yubikey) could use other channels, e.g. twitter or people they know, to communicate ahead of time that they are going to replace their secondary proof. All the while, their email and RubyGems account (primary proof) wouldn't change. I know this is veering into protocol territory, so may be out of scope for this PR. But just wanted to mention it, since I like the overall idea of not using a third-party IdP. However, I also think the risk of account takeovers is real and could use more thought. This is one idea on how to reduce that risk. |
Rendered RFC.
We'd like to propose a new gem signing mechanism. We believe the new mechanism will be easier to use and more secure. Our goal is that eventually, almost all gems are signed as a matter of course. The proposed design builds on sigstore, an OpenSSF-backed project for software signatures.
As part of our work, we have developed a proof of concept system that works as a gem plugin.
We have drafted the RFC for an audience who are familiar with RubyGems, but who may not be familiar with sigstore or with the security primitives it utilizes. We note that the reference-level guide section is very detailed, and most of it can be safely skipped on a first reading.
We will be available on the Bundler Slack if you wish to discuss there.
Contributors: @bettymakes, @aellispierce, @jchestershopify, @jenshenny, @tomstuart, @doodzik and @rochlefebvre.
Special thanks to: @dlorenc, @lukehinds and @bobcallaway for help on sigstore questions.