DigiCert in-addr.arpa X.509 certificate mis-issuance

Possible DigiCert in-addr.arpa Mis-issuance Cynthia Revström 26.02.19 11:02
Hello dev.security.policy

Apologies if I have made any mistakes in how I post, this is my first

time posting here. Anyway:

I have managed to issue a certificate with a FQDN in the SAN that I do

not have control of via Digicert.

The precert is here: https://crt.sh/?id=1231411316

SHA256: 651B68C520492A44A5E99A1D6C99099573E8B53DEDBC69166F60685863B390D1

I have notified Digicert who responded back with a generic response

followed by the certificate being revoked through OCSP. However I

believe that this should be wider investigated, since this cert was

issued by me adding 69.168.110.79.in-addr.arpa to my SAN, a DNS area

that I do control though reverse DNS.

When I verified 5.168.110.79.in-addr.arpa (same subdomain), I noticed

that the whole of in-addr.arpa became validated on my account, instead

of just my small section of it (168.110.79.in-addr.arpa at best).

To test if digicert had just in fact mis-validated a FQDN, I tested with

the reverse DNS address of 192.168.1.1, and it worked and Digicert

issued me a certificate with 1.1.168.192.in-addr.arpa on it.

Is there anything else dev.security.policy needs to do with this? This

seems like a clear case of mis issuance. It’s also not clear if

in-addr.arpa should even be issuable.

I would like to take a moment to thank Ben Cartwright-Cox and igloo22225

in pointing out this violation.

Regards

Cynthia Revström

Re: Possible DigiCert in-addr.arpa Mis-issuance Jeremy Rowley 26.02.19 11:05 Re: Possible DigiCert in-addr.arpa Mis-issuance Matthew Hardeman 26.02.19 15:11
Is it even proper to have a SAN dnsName in in-addr.arpa ever?

While in-addr.arpa IS a real DNS heirarchy under the .arpa TLD, it rarely

has anything other than PTR and NS records defined.

Here this was clearly achieved by creating a CNAME record for

69.168.110.79.in-addr.arpa pointed to cynthia.re.

I’ve never seen any software or documentation anywhere attempting to

utilize a reverse-IP formatted in-addr.arpa address as though it were a

normal host name for resolution.  I wonder whether this isn’t a case that

should just be treated as an invalid domain for purposes of SAN dnsName

(like .local).

Re: Possible DigiCert in-addr.arpa Mis-issuance Cynthia Revström 26.02.19 15:17 RE: Possible DigiCert in-addr.arpa Mis-issuance Jeremy Rowley 26.02.19 16:55
>From our side, a validation agent weirdly scoped the domain, saying that the domain was approved using an email to admin@in-addr.arpa. However, the email clearly went to admin@5.168.110.79.in-addr.arpa. We’re looking to see 1) how did the validation staff override the domain approval scope and 2) did anyone in validation ever do this before. Once we complete that search, we’ll be able to file a bug report and give you more information on what exactly went wrong.

.arpa is in IANA’s root zone per https://www.iana.org/domains/root/db.

Jeremy

https://lists.mozilla.org/listinfo/dev-security-policy

Re: Possible DigiCert in-addr.arpa Mis-issuance Jakob Bohm 27.02.19 06:54

On 27/02/2019 00:10, Matthew Hardeman wrote:

> Is it even proper to have a SAN dnsName in in-addr.arpa ever?

>

> While in-addr.arpa IS a real DNS heirarchy under the .arpa TLD, it rarely

> has anything other than PTR and NS records defined.

>

While there is no current use, and the test below was obviously

somewhat contrived (and seems to have triggered a different issue),

one cannot rule out the possibility of a need appearing in the

future.

One hypothetical use would be to secure BGP traffic, as certificates

with IpAddress SANs are less commonly supported.

Enjoy

Jakob



Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com

Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10

This public discussion message is non-binding and may contain errors.

WiseMo – Remote Service Management for PCs, Phones and Embedded

Re: Possible DigiCert in-addr.arpa Mis-issuance Nick Lamb 27.02.19 07:04
On Tue, 26 Feb 2019 17:10:49 -0600

Matthew Hardeman via dev-security-policy

<dev-secur…@lists.mozilla.org> wrote:

> Is it even proper to have a SAN dnsName in in-addr.arpa ever?

It does feel as though ARPA should consider adding a CAA record to

in-addr.arpa and similar hierarchies that don’t want certificates,

denying all CAs, as a defence in depth measure.

Nick.

Re: Possible DigiCert in-addr.arpa Mis-issuance Ryan Sleevi 27.02.19 07:34
Alternatively, and perhaps more comprehensively, it may be better to ensure

that those Special Use Domains that are either delegated to or reserved by

IANA or the IESG can only have certificates issued by those respective

organizations.

These are enumerated prosaically at

https://www.iana.org/domains/reserved for those reserved by policy, and

https://www.iana.org/assignments/special-use-domain-names/special-use-domain-names.xhtml

is a registry of those reserved by relevant standards.

This approach is already taken with regards to IP addresses, and more

comprehensively avoids ambiguity. It has the benefit of defaulting secure –

by not requiring a domain holder (including IANA) to somehow take special

action to protect existing practice. Should concrete use cases present

themselves – of which BGP is not one (see BGPsec for more details) – then

those can be relaxed on a case by case basis. The .onion Domain is an

example of a pre-existing relaxation.

This would still permit .arpa certificates – specific language would be

needed (and should be done) to either prohibit or apply the same

consistency to as IP certificates – but would otherwise close a class of

“obvious” errors. The suggestion was intentionally not a blanket ban, as

IANA/ICANN does and has obtained legitimate certificates for the example

domains in the past.

RE: Possible DigiCert in-addr.arpa Mis-issuance Tim Hollebeek 27.02.19 07:43


> On 27/02/2019 00:10, Matthew Hardeman wrote:

> > Is it even proper to have a SAN dnsName in in-addr.arpa ever?

> >

> > While in-addr.arpa IS a real DNS heirarchy under the .arpa TLD, it

> > rarely has anything other than PTR and NS records defined.

> >

>

> While there is no current use, and the test below was obviously somewhat

> contrived (and seems to have triggered a different issue), one cannot rule

> out

> the possibility of a need appearing in the future.

At least the last time this came up a few years ago, there were actually a

significant number of webservers running under in-addr.arpa, with Comodo and

LE certificates (as well as a handful of others).  I believe Corey posted a

list.

Exactly what they were doing there is an open question, and when I asked, no

one responded.  I’m still very curious as to why some people seem to actually

be running servers there, or if it’s just a side-effect of misconfigured

CNAMEs causing them to appear to be there, or similar.

-Tim

Re: Possible DigiCert in-addr.arpa Mis-issuance Cynthia Revström 27.02.19 07:45
Okay that seems like an issue as to me that says that this could have

happened to any domain and not just in-addr.arpa?

– Cynthia


On 2019-02-27 01:55, Jeremy Rowley via dev-security-policy wrote:

>  From our side, a validation agent weirdly scoped the domain, saying that the domain was approved using an email to admin@in-addr.arpa. However, the email clearly went to admin@5.168.110.79.in-addr.arpa. We’re looking to see 1) how did the validation staff override the domain approval scope and 2) did anyone in validation ever do this before. Once we complete that search, we’ll be able to file a bug report and give you more information on what exactly went wrong.

>

> .arpa is in IANA’s root zone per https://www.iana.org/domains/root/db.

>

> Jeremy

>

> —–Original Message—–

> From: dev-security-policy <dev-security-…@lists.mozilla.org> On Behalf Of Cynthia Revström via dev-security-policy

> Sent: Tuesday, February 26, 2019 4:17 PM

> To: Matthew Hardeman <mhar…@gmail.com>

> Cc: dev-secur…@lists.mozilla.org; b…@benjojo.co.uk

> Subject: Re: Possible DigiCert in-addr.arpa Mis-issuance

>

> I am not so sure that is proper to have .arpa domains in the SANs.

>

> But I think the larger issue I think is that this might allow for non in-addr.arpa domains to be used as well.

>

> It was just that I just tried to get a cert for my domain for a test to see if that would be issued.

>

> And upon verifying the domain via email, I saw that on the page linked in the email it had an option along the lines of “Authorize in-addr.arpa for all orders on account #123456” (not that exact text but the same meaning).

>

> Now I am not sure if this would work, but I suspect if for example I got DNS control over cynthia.site.google.com, I could get a cert for google.com if this is a systemic issue and not just a one off.

>

> – Cynthia

>

> On 2019-02-27 00:10, Matthew Hardeman wrote:

>> Is it even proper to have a SAN dnsName in in-addr.arpa ever?

>>

>> While in-addr.arpa IS a real DNS heirarchy under the .arpa TLD, it

>> rarely has anything other than PTR and NS records defined.

>>

>> Here this was clearly achieved by creating a CNAME record for

>> 69.168.110.79.in-addr.arpa pointed to cynthia.rehttp://cynthia.re>.

>>

>> I’ve never seen any software or documentation anywhere attempting to

>> utilize a reverse-IP formatted in-addr.arpa address as though it were

>> a normal host name for resolution.  I wonder whether this isn’t a case

>> that should just be treated as an invalid domain for purposes of SAN

>> dnsName (like .local).

>>

>> On Tue, Feb 26, 2019 at 1:05 PM Jeremy Rowley via dev-security-policy

>> <dev-secur…@lists.mozilla.org

>> <mailto:dev-secur…@lists.mozilla.org>> wrote:

>>

>>      Thanks Cynthia. We are investigating and will report back shortly.

>>      ________________________________

>>      From: dev-security-policy

>>      <dev-security-…@lists.mozilla.org

>>      dev-security-policy-bounces@lists.mozilla.org>> on behalf

>>      of Cynthia Revström via dev-security-policy

>>      <dev-secur…@lists.mozilla.org

>>      dev-secur…@lists.mozilla.org>>

>>      Sent: Tuesday, February 26, 2019 12:02:20 PM

>>      To: dev-secur…@lists.mozilla.org

>>      dev-secur…@lists.mozilla.org>

>>      Cc: b…@benjojo.co.ukb…@benjojo.co.uk>

RE: Possible DigiCert in-addr.arpa Mis-issuance Jeremy Rowley 27.02.19 07:48
Well yes, assuming the validation person overrides the safeguards and randomly changes the scope of the domain validation to something other than what the system specifies. We’re still looking into how this happened and if the validation agent did this in any other case.


—–Original Message—–

From: dev-security-policy <dev-security-…@lists.mozilla.org> On Behalf Of Cynthia Revström via dev-security-policy

Re: Possible DigiCert in-addr.arpa Mis-issuance Matthew Hardeman 27.02.19 08:53

On Wed, Feb 27, 2019 at 9:04 AM Nick Lamb <n…@tlrmx.org> wrote:

>

> It does feel as though ARPA should consider adding a CAA record to

> in-addr.arpa and similar hierarchies that don’t want certificates,

> denying all CAs, as a defence in depth measure.

>

Unless I significantly misunderstand CAA, this mechanism would not

necessarily be effective.

The normal mode of operation is that at the in-addr.arpa zone delegates sub

zones, for example 199.in-addr.arpa to the relevant RIR via an NS record.

Further, the relevant RIR would delegate sub zones of that zone via NS

records to an IP space holder, for example 88.99.199.in-addr.arpa would

have NS records configured on the RIR name servers which would refer to the

authoritative DNS servers serving the IP space holder for 199.99.88.0/24.

As such, superseding CAA records which would allow issuance could be added

back into those hierarchies by the DNS admins of those zones.

Re: Possible DigiCert in-addr.arpa Mis-issuance Corey Bonnell 27.02.19 19:05
Hi Tim,

As you said, I vaguely recall this coming up in some discussion (perhaps in the CAB Forum Validation Subcommittee?) but nothing was concluded. I believe the list was merely a crt.sh query of all unexpired certificates with a dNSName ending in “in-addr.arpa”: https://crt.sh/?dNSName=%25.in-addr.arpa&exclude=expired

The query results are definitely worth a look as there are some unexpected findings, such as wildcards (such as “*.0.195.206.in-addr.arpa”) and host nodes (such as “www.175.232.77.in-addr.arpa”, etc.) under in-addr.arpa. Several of the domain names starting with “www” actually appear to resolve to an IP address with a web server running. Definitely an interesting (ab)use of the reverse zones.

Thanks,

Corey

RE: Possible DigiCert in-addr.arpa Mis-issuance Jeremy Rowley 27.02.19 21:52
Hi Cynthia,

We’ve figured out what happened with your certificate but are still looking at whether other certificates were issued using the same process. I don’t have enough information to file an incident report yet, but I wanted to give you the update I promised earlier.

Basically, here’s what happened:

1. A validation agent took an email address provided during a chat session with you and set that email as a WHOIS admin email address. This email qualified as a constructed email (admin@domain)

2. The system marked the WHOIS as unavailable for automated parsing (generally, this happens if we are being throttled or the WHOIS info is behind a CAPTCHA), which allows a validation agent to manually upload a WHOIS document

3. The WHOIS document uploaded was a screen capture of WHOIS information that did not match the domain being requested but was close enough that the mis-match went unnoticed.

4. The validation agent specified the approval scope as id-addr.arpa which is normal for a domain approved by the admin listed in WHOIS. As a constructed email, the approval scope should have been limited to the scope set by the constructed address.

5. The validation agent specified that the email address listed in WHOIS matched the email address provided by you (admin@domain) and sent the email to authorize the legit cert

6 . When the validation completed successfully, the validation authorized your account for all certificates with the in-addr.arpa domain. This was because the scope was improperly set based on the validation agent’s improper specification of the source of the email address.

7. During the review, no one noticed that the WHOIS document did not match the verification email nor did anyone notice that the email used for verification was actually a constructed email instead of the WHOIS admin email.

8. The mis-issued certificate essentially “piggy-backed” on the previous verification because of the erroneous approval scope set by the validation staff.

Make sense?

We shut down manual WHOIS verification while we figure out how to improve the process and eliminate validation mistakes like this. We’ll post another update after we figure out how to improve the process and after the data review is complete.


Jeremy

—–Original Message—–

From: dev-security-policy <dev-security-…@lists.mozilla.org> On Behalf Of Cynthia Revström via dev-security-policy

Sent: Tuesday, February 26, 2019 4:17 PM

To: Matthew Hardeman <mhar…@gmail.com>

Cc: dev-secur…@lists.mozilla.org; b…@benjojo.co.uk

Subject: Re: Possible DigiCert in-addr.arpa Mis-issuance

I am not so sure that is proper to have .arpa domains in the SANs.

But I think the larger issue I think is that this might allow for non in-addr.arpa domains to be used as well.

It was just that I just tried to get a cert for my domain for a test to see if that would be issued.

And upon verifying the domain via email, I saw that on the page linked in the email it had an option along the lines of “Authorize in-addr.arpa for all orders on account #123456” (not that exact text but the same meaning).

Now I am not sure if this would work, but I suspect if for example I got DNS control over cynthia.site.google.com, I could get a cert for google.com if this is a systemic issue and not just a one off.

– Cynthia

On 2019-02-27 00:10, Matthew Hardeman wrote:

> Is it even proper to have a SAN dnsName in in-addr.arpa ever?

>

> While in-addr.arpa IS a real DNS heirarchy under the .arpa TLD, it

> rarely has anything other than PTR and NS records defined.

>

Re: Possible DigiCert in-addr.arpa Mis-issuance Nick Lamb 28.02.19 03:21
On Thu, 28 Feb 2019 05:52:14 +0000

Jeremy Rowley via dev-security-policy

dev-secur…@lists.mozilla.org> wrote:

Hi Jeremy,


> 4. The validation agent specified the approval scope as id-addr.arpa

I assume this is a typo by you not the agent, for in-addr.arpa ?

Meanwhile, and without prejudice to the report itself once made:


> 2. The system marked the WHOIS as unavailable for automated parsing

> (generally, this happens if we are being throttled or the WHOIS info

> is behind a CAPTCHA), which allows a validation agent to manually

> upload a WHOIS document

This is a potentially large hole in issuance checks based on WHOIS.

Operationally the approach taken (“We can’t get it to work, press on”)

makes sense, but if we take a step back there’s obvious potential for

nasty security surprises like this one.

There has to be something we can do here, I will spitball something in

a next paragraph just to have something to start with, but to me if it

turns out we can’t improve on basically “sometimes it doesn’t work so

we just shrug and move on” we need to start thinking about deprecating

this approach altogether. Not just for DigiCert, for everybody.

– Spitball: What if the CA/B went to the registries, at least the big

  ones, and said we need this, strictly for this defined purpose, give

  us either reliable WHOIS, or RDAP, or direct database access or

  _something_ we can automate to do these checks ? The nature of CA/B

  may mean that it’s not appropriate to negotiate paying for this

  (pressuring suppliers to all agree to offer members the same rates is

  just as much a problem as all agreeing what you’ll charge customers)

  but it should be able to co-ordinate making sure members get access,

  and that it isn’t opened up to dubious data resellers that the

  registries don’t want rifling through their database.

My argument to the registries would be that this is a service for their

customers. Unlike the data resellers, either the registry customer, or

some agent of theirs is asking you to authenticate their registration,

so giving you access makes sense as part of what the registry does for

its customers anyway.


> 7. During the review, no one noticed that the WHOIS document did not

> match the verification email nor did anyone notice that the email

> used for verification was actually a constructed email instead of the

> WHOIS admin email

So, reviews are good, but this review was not very effective. Valuable

to consider in the final report why not and how that can be improved.

Just to be clear though, are you sure “no one noticed” ? It can happen

that in review processes somebody does notice the issue, but they

are persuaded or persuade themselves that it’s fine. A British railway

incident occurred when the person transcribing a document effectively

“moved” a railway crossing. Manual reviewers did see it, and so did the

controllers responsible for managing the crossing, but both persuaded

themselves that the movement must be a correction and approved it.

With the crossing now shown in the wrong place, instructions authorising

use of the crossing were no longer protected by the controller’s view

of the movement of trains, this resulted in a “near miss” and thanks to

the victim’s persistence in demanding it be properly investigated

fortunately accident investigators visited the crossing, found the

mistake and had things corrected before anyone died.

Nick.

Re: Possible DigiCert in-addr.arpa Mis-issuance Ryan Sleevi 28.02.19 03:43
Unfortunately, this is not really viable. The CA/Browser Forum maintains

relationships with ICANN, as do individual members. While this, on its

face, seems reasonable, there are practical, business, and legal concerns

that prevent this from being viable. Further, proposals which would require

membership in the CA/Browser Forum should, on their face, be rejected – a

CA should not have to join the Forum in order to be a CA.

I do agree, however, that the use of WHOIS data continues to show

problematic incidents – whether it’s with OCR issues or manual entry – and

suspect a more meaningful solution is to move away from this model

entirely. The recently approved methods to the BRs for expressing contact

information via the DNS directly is one such approach. The GDPR issues

surrounding WHOIS and RDAP have already led it to be compelling in its own

right.

Most importantly, you are on the right path of questions, though – which is

we should examine such incidents systemically and look for improvements,

and not convince ourselves that the status quo is the best possible

solution 🙂

Re: Possible DigiCert in-addr.arpa Mis-issuance Daniel McCarney 28.02.19 05:22

>

> I believe the list was merely a crt.sh query of all unexpired certificates

> with a dNSName ending in “in-addr.arpa”:

> https://crt.sh/?dNSName=%25.in-addr.arpa&exclude=expired

Any list for this general issue should also consider unexpired certificates

with a dNSName ending in “ip6.arpa” to cover the IPv6 reverse zone in

addition to the IPv4 one. I noticed there are similar interesting

wildcards/host nodes under the ip6.arpa zone when I was writing a linter[0]

for this.

[0] – https://github.com/zmap/zlint/pull/260


On Wed, Feb 27, 2019 at 10:05 PM Corey Bonnell via dev-security-policy <

dev-secur…@lists.mozilla.org> wrote:

> On Wednesday, February 27, 2019 at 10:43:15 AM UTC-5, Tim Hollebeek wrote:

> > > On 27/02/2019 00:10, Matthew Hardeman wrote:

> > > > Is it even proper to have a SAN dnsName in in-addr.arpa ever?

> > > >

> > > > While in-addr.arpa IS a real DNS heirarchy under the .arpa TLD, it

> > > > rarely has anything other than PTR and NS records defined.

> > > >

> > >

Re: Possible DigiCert in-addr.arpa Mis-issuance Matthew Hardeman 28.02.19 16:04
In addition to the GDPR concerns over WHOIS and RDAP data, reliance upon

these data sources has a crucial differentiation from other domain

validation methods.

Specifically, the WHOIS/RDAP data sources are entirely “off-path” with

respect to how a browser will locate and access a given site.  To my way of

thinking, this renders these mechanisms functionally inferior to an

“on-path” mechanism, such as reliances upon demonstrated change control

over an authoritative DNS record or even demonstration content change

control over a website.

Since domain validation is, in theory, about validating that the party to

whom a certificate is to be issued has demonstrated control over the

subject of the desired name(s) or the name space of the desired name(s), it

seems clear that “off-path” validation is less valuable as a security

measure.

Although I’m aware that the BRs bless a number of methods, it’s also clear

that methods have been excluded by the Mozilla root program before.  Is it

time to consider further winnowing down the accepted methods?

On Thu, Feb 28, 2019 at 5:43 AM Ryan Sleevi via dev-security-policy <

dev-secur…@lists.mozilla.org> wrote:

Re: Possible DigiCert in-addr.arpa Mis-issuance Matthew Hardeman 28.02.19 17:28

On Wednesday, February 27, 2019 at 8:54:35 AM UTC-6, Jakob Bohm wrote:

> One hypothetical use would be to secure BGP traffic, as certificates

> with IpAddress SANs are less commonly supported.

The networking / interconnection world has already worked out the trust hierarchy for the RPKI scheme.  As there are a number of global RIRs who are the authoritative source of ASN and IP space information, they’ve elected to themselves be the Root CAs involved.  Its an interesting infrastructure.  You can learn more about it here:

https://www.arin.net/resources/rpki/index.html

Re: Possible DigiCert in-addr.arpa Mis-issuance Jakob Bohm 01.03.19 04:40

On 01/03/2019 01:04, Matthew Hardeman wrote:

> In addition to the GDPR concerns over WHOIS and RDAP data, reliance upon

> these data sources has a crucial differentiation from other domain

> validation methods.

>

> Specifically, the WHOIS/RDAP data sources are entirely “off-path” with

> respect to how a browser will locate and access a given site.  To my way of

> thinking, this renders these mechanisms functionally inferior to an

> “on-path” mechanism, such as reliances upon demonstrated change control

> over an authoritative DNS record or even demonstration content change

> control over a website.

>

> Since domain validation is, in theory, about validating that the party to

> whom a certificate is to be issued has demonstrated control over the

> subject of the desired name(s) or the name space of the desired name(s), it

> seems clear that “off-path” validation is less valuable as a security

> measure.

>

And there, you completely misunderstood the purpose.  The purpose of

domain validation is to determine if the certificate application is

(directly or indirectly) authorized by the domain registrant.

 Technical control (via the various automated methods) is a proxy to

this information, but not as close to the truth as WHOIS validations

(provided either is done securely).

The panic mishandling of GDPR concerns by ICANN (something that

continues in the current process to make a “permanent” solution) has

essentially crippled all useful purposes of the WHOIS/RDAP database.

Including making it near impossible to do domain ownership (as opposed

to mere control) validation impossible except for a few national

TLDs that continue to provide open WHOIS.

I sincerely hope that ICANN cleans up its misunderstandings and

reopens WHOIS, at least for domain owners that request this (it

currently can’t be done because registrars are vary of implementing

the temporary specification of how to do that, as a new spec is in

the works).

Therefore WHOIS validation should remain valid, but only if the

actual registry/registrar provides authoritative computer readable

data for the domain in question.  (Thus having to do OCR or similar

of a picture of text is not acceptable, and neither are manually

entered entries).

In particular “substitute WHOIS” lookups via manual steps that

don’t result in the validation computers directly reading the data

from the registrar/registry server should not be allowed.

There are many ways this can be done technically, ranging from a

straightforward WHOIS lookup from multiple network vantage points

to a process where a special web browser is used to authenticate

through CAPTCHAs etc. until the validation system automatically

captures and parses the “known correct” textual web response

from a known registrar/registry url.

This in turn means that IV, OV and EV certs will often need to use

other means of associating the domain control with the certificate

applicant entity.  For example one or more challenge values/pins

can be securely provided to the real world entity validated, then

used in the protocol for validating domain control.  But this still

only validates domain control, not legitimacy of domain authority.

Re: Possible DigiCert in-addr.arpa Mis-issuance Wayne Thayer 01.03.19 09:00
https://bugzilla.mozilla.org/show_bug.cgi?id=1531817 has been created to

track this issue.


On Wed, Feb 27, 2019 at 10:52 PM Jeremy Rowley via dev-security-policy <

dev-secur…@lists.mozilla.org> wrote:

> Hi Cynthia,

>

> We’ve figured out what happened with your certificate but are still

> looking at whether other certificates were issued using the same process. I

> don’t have enough information to file an incident report yet, but I wanted

> to give you the update I promised earlier.

>

> Basically, here’s what happened:

> 1. A validation agent took an email address provided during a chat session

> with you and set that email as a WHOIS admin email address. This email

> qualified as a constructed email (admin@domain)

> 2. The system marked the WHOIS as unavailable for automated parsing

> (generally, this happens if we are being throttled or the WHOIS info is

> behind a CAPTCHA), which allows a validation agent to manually upload a

> WHOIS document

> 3. The WHOIS document uploaded was a screen capture of WHOIS information

> that did not match the domain being requested but was close enough that the

> mis-match went unnoticed.

> 4. The validation agent specified the approval scope as id-addr.arpa which

> is normal for a domain approved by the admin listed in WHOIS. As a

> constructed email, the approval scope should have been limited to the scope

> set by the constructed address.

> 5. The validation agent specified that the email address listed in WHOIS

> matched the email address provided by you (admin@domain) and sent the

> email to authorize the legit cert

> 6 . When the validation completed successfully, the validation authorized

> your account for all certificates with the in-addr.arpa domain. This was

> because the scope was improperly set based on the validation agent’s

> improper specification of the source of the email address.

> 7. During the review, no one noticed that the WHOIS document did not match

> the verification email nor did anyone notice that the email used for

> verification was actually a constructed email instead of the WHOIS admin

> On 2019-02-27 00:10, Matthew Hardeman wrote:

> > Is it even proper to have a SAN dnsName in in-addr.arpa ever?

> >

> > While in-addr.arpa IS a real DNS heirarchy under the .arpa TLD, it

> > rarely has anything other than PTR and NS records defined.

> >

> > Here this was clearly achieved by creating a CNAME record for

> > 69.168.110.79.in-addr.arpa pointed to cynthia.re <http://cynthia.re>.

> >

> > I’ve never seen any software or documentation anywhere attempting to

> > utilize a reverse-IP formatted in-addr.arpa address as though it were

> > a normal host name for resolution.  I wonder whether this isn’t a case

> > that should just be treated as an invalid domain for purposes of SAN

> > dnsName (like .local).

> >

> > On Tue, Feb 26, 2019 at 1:05 PM Jeremy Rowley via dev-security-policy

> > dev-secur…@lists.mozilla.org

> >     _______________________________________________

> >     dev-security-policy mailing list

> >     dev-secur…@lists.mozilla.org

> >     dev-secur…@lists.mozilla.org>

> >     https://lists.mozilla.org/listinfo/dev-security-policy

> >     _______________________________________________

> >     dev-security-policy mailing list

> >     dev-secur…@lists.mozilla.org

> >     dev-secur…@lists.mozilla.org>

> >     https://lists.mozilla.org/listinfo/dev-security-policy

RE: Possible DigiCert in-addr.arpa Mis-issuance Jeremy Rowley 01.03.19 09:07
Thanks Wayne


 

From: Wayne Thayer <wth…@mozilla.com>

Sent: Friday, March 1, 2019 10:00 AM

To: Jeremy Rowley <jeremy…@digicert.com>

Cc: mozilla-dev-security-policy <mozilla-dev-s…@lists.mozilla.org>

Subject: Re: Possible DigiCert in-addr.arpa Mis-issuance

 

https://bugzilla.mozilla.org/show_bug.cgi?id=1531817 has been created to track this issue.

 

On Wed, Feb 27, 2019 at 10:52 PM Jeremy Rowley via dev-security-policy <dev-secur…@lists.mozilla.org <mailto:dev-secur…@lists.mozilla.org> > wrote:

Hi Cynthia,

We’ve figured out what happened with your certificate but are still looking at whether other certificates were issued using the same process. I don’t have enough information to file an incident report yet, but I wanted to give you the update I promised earlier.

Basically, here’s what happened:

1. A validation agent took an email address provided during a chat session with you and set that email as a WHOIS admin email address. This email qualified as a constructed email (admin@domain)

2. The system marked the WHOIS as unavailable for automated parsing (generally, this happens if we are being throttled or the WHOIS info is behind a CAPTCHA), which allows a validation agent to manually upload a WHOIS document

3. The WHOIS document uploaded was a screen capture of WHOIS information that did not match the domain being requested but was close enough that the mis-match went unnoticed.

4. The validation agent specified the approval scope as id-addr.arpa which is normal for a domain approved by the admin listed in WHOIS. As a constructed email, the approval scope should have been limited to the scope set by the constructed address.

5. The validation agent specified that the email address listed in WHOIS matched the email address provided by you (admin@domain) and sent the email to authorize the legit cert

6 . When the validation completed successfully, the validation authorized your account for all certificates with the in-addr.arpa domain. This was because the scope was improperly set based on the validation agent’s improper specification of the source of the email address.

7. During the review, no one noticed that the WHOIS document did not match the verification email nor did anyone notice that the email used for verification was actually a constructed email instead of the WHOIS admin email.

8. The mis-issued certificate essentially “piggy-backed” on the previous verification because of the erroneous approval scope set by the validation staff.

Make sense?

We shut down manual WHOIS verification while we figure out how to improve the process and eliminate validation mistakes like this. We’ll post another update after we figure out how to improve the process and after the data review is complete.

Jeremy

—–Original Message—–

From: dev-security-policy <dev-security-…@lists.mozilla.org <mailto:dev-security-policy-bounces@lists.mozilla.org> > On Behalf Of Cynthia Revström via dev-security-policy

Sent: Tuesday, February 26, 2019 4:17 PM

To: Matthew Hardeman <mhar…@gmail.com <mailto:mhar…@gmail.com> >

Cc: dev-secur…@lists.mozilla.org <mailto:dev-secur…@lists.mozilla.org> ; b…@benjojo.co.uk <mailto:b…@benjojo.co.uk>

Subject: Re: Possible DigiCert in-addr.arpa Mis-issuance

I am not so sure that is proper to have .arpa domains in the SANs.

But I think the larger issue I think is that this might allow for non in-addr.arpa domains to be used as well.

It was just that I just tried to get a cert for my domain for a test to see if that would be issued.

And upon verifying the domain via email, I saw that on the page linked in the email it had an option along the lines of “Authorize in-addr.arpa for all orders on account #123456” (not that exact text but the same meaning).

Now I am not sure if this would work, but I suspect if for example I got DNS control over cynthia.site.google.comhttp://cynthia.site.google.com> , I could get a cert for google.comhttp://google.com>  if this is a systemic issue and not just a one off.


– Cynthia

On 2019-02-27 00:10, Matthew Hardeman wrote:

> Is it even proper to have a SAN dnsName in in-addr.arpa ever?

>

> While in-addr.arpa IS a real DNS heirarchy under the .arpa TLD, it

> rarely has anything other than PTR and NS records defined.

>

> Here this was clearly achieved by creating a CNAME record for

> 69.168.110.79.in-addr.arpa pointed to cynthia.rehttp://cynthia.re>  http://cynthia.re>.

>

> I’ve never seen any software or documentation anywhere attempting to

> utilize a reverse-IP formatted in-addr.arpa address as though it were

> a normal host name for resolution.  I wonder whether this isn’t a case

> that should just be treated as an invalid domain for purposes of SAN

> dnsName (like .local).

>

> On Tue, Feb 26, 2019 at 1:05 PM Jeremy Rowley via dev-security-policy

> <dev-secur…@lists.mozilla.org <mailto:dev-secur…@lists.mozilla.org>

> <mailto:dev-secur…@lists.mozilla.org <mailto:dev-secur…@lists.mozilla.org> >> wrote:

>

>     Thanks Cynthia. We are investigating and will report back shortly.

>     ________________________________

>     From: dev-security-policy

>     <dev-security-…@lists.mozilla.org <mailto:dev-security-policy-bounces@lists.mozilla.org>

>     dev-security-policy-bounces@lists.mozilla.orgdev-security-policy-bounces@lists.mozilla.org> >> on behalf

>     of Cynthia Revström via dev-security-policy

>     <dev-secur…@lists.mozilla.org <mailto:dev-secur…@lists.mozilla.org>

>     dev-secur…@lists.mozilla.orgdev-secur…@lists.mozilla.org> >>

>     Sent: Tuesday, February 26, 2019 12:02:20 PM

>     To: dev-secur…@lists.mozilla.org <mailto:dev-secur…@lists.mozilla.org>

>     dev-secur…@lists.mozilla.orgdev-secur…@lists.mozilla.org> >

>     Cc: b…@benjojo.co.ukb…@benjojo.co.uk>  b…@benjojo.co.ukb…@benjojo.co.uk> >

>     dev-secur…@lists.mozilla.orgdev-secur…@lists.mozilla.org> >

>     https://lists.mozilla.org/listinfo/dev-security-policy

>     _______________________________________________

>     dev-security-policy mailing list

>     dev-secur…@lists.mozilla.org <mailto:dev-secur…@lists.mozilla.org>

>     dev-secur…@lists.mozilla.orgdev-secur…@lists.mozilla.org> >

>     https://lists.mozilla.org/listinfo/dev-security-policy

>

_______________________________________________

dev-security-policy mailing list

dev-secur…@lists.mozilla.org <mailto:dev-secur…@lists.mozilla.org>

https://lists.mozilla.org/listinfo/dev-security-policy

_______________________________________________

dev-security-policy mailing list

dev-secur…@lists.mozilla.orgdev-secur…@lists.mozilla.org>

https://lists.mozilla.org/listinfo/dev-security-policy

Re: Possible DigiCert in-addr.arpa Mis-issuance George Macon 01.03.19 16:49

On 2/28/19 12:52 AM, Jeremy Rowley wrote:

> 4. The validation agent specified the approval scope as id-addr.arpa which is normal for a domain approved by the admin listed in WHOIS. As a constructed email, the approval scope should have been limited to the scope set by the constructed address.

One specific question on this point: Why did the software permit setting

the approval scope to a public suffix (as defined by inclusion on the

public suffix list)? Could validation agent action set the approval

scope to some other two-label public suffix like co.uk?

-George Macon

Re: Possible DigiCert in-addr.arpa Mis-issuance Cynthia Revström 02.03.19 00:45

On 2019-03-02 01:49, George Macon via dev-security-policy wrote:

> One specific question on this point: Why did the software permit setting

> the approval scope to a public suffix (as defined by inclusion on the

> public suffix list)? Could validation agent action set the approval

> scope to some other two-label public suffix like co.uk?

I think this is highly unlikely seeing as this was a human error and

unlike in-addr.arpa, people might know about .co.uk.

– Cynthia

Products You May Like

Leave a Reply

Your email address will not be published. Required fields are marked *