babylonhealth / certificate-transparency-android Goto Github PK

If I read this article correctly I believe it means jcenter is going away.

"sunsetting Bintray (including JCenter), GoCenter, and ChartCenter services on May 1st, 2021."

https://www.infoq.com/news/2021/02/jfrog-jcenter-bintray-closure/

Would you consider publishing to maven central or another suitable repository?

Starting from May 1, 2021 JCenter will stop accepting updates as its being sunsetted.

The artifacts for this project should be deployed to an alternate repository such as MavenCentral.

Thanks!

Error:

Caused by: java.lang.NoClassDefFoundError: Failed resolution of: Lcom/babylon/certificatetransparency/CTInterceptorBuilderExtKt;

 Caused by: java.lang.ClassNotFoundException: Didn't find class "com.babylon.certificatetransparency.CTInterceptorBuilderExtKt" on path: DexPathList[[zip file]]
    at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:207)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:379)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:312)
    	... 60 more

When setting up this library, it's mandatory to set at-least one host to set CT for. Now lets say we have wrongly configured host name (Could be wrong regex or a typo) in includeHost() will cause the enabledForCertificateTransparency(String) to fail and thus return a success (VerificationResult.Success.DisabledForHost)

Eg:

.hostnameVerifier(certificateTransparencyHostnameVerifier(
    OkHostnameVerifier.INSTANCE
) {
     +"github.con" // wrong hostname, by mistaked added .co**n** instead of .co**m**
})

Will cause all the calls to github.com to succeed.

Suggestion to resolve

• Make the default as CT enabled for all the hosts

This way people with basic usecase do not need to do additional configuration and this mitigating the issue.

Report errors to report-uri whether or not fail on error is enabled

My certificate is about to expire and and on the last few days, some instances of my application are having problems connecting to the server. They just don't connect. Even on the same device, sometimes after installing the app it works fine, other times it just doesn't connect.

Digging in I discovered that removing this library fixes the issue. Any tip on how to fix this?

Or any ideas why this is happening?

As per comments in square/okhttp#2938, provide clients the ability to use their own OkHttp client. This would allow, amongst other things, for proxy server settings to be set etc.

I met a build time issue with class duplicated with kotlinx-coroutines-core-jvm and kotlinx-coroutines-core. And I list the dependencies

./gradlew :app:dependencies

|    |    |    +--- com.babylon.certificatetransparency:certificatetransparency-android:0.3.0
|    |    |    |    +--- com.babylon.certificatetransparency:certificatetransparency:0.3.0
|    |    |    |    |    +--- org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.4.0
|    |    |    |    |    |    +--- org.jetbrains.kotlin:kotlin-stdlib:1.4.0 (*)
|    |    |    |    |    |    \--- org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.4.0 (*)
|    |    |    |    |    +--- org.bouncycastle:bcpkix-jdk15to18:1.66
|    |    |    |    |    |    \--- org.bouncycastle:bcprov-jdk15to18:1.66
|    |    |    |    |    +--- org.bouncycastle:bcprov-jdk15to18:1.66
|    |    |    |    |    +--- org.bouncycastle:bctls-jdk15to18:1.66
|    |    |    |    |    |    \--- org.bouncycastle:bcprov-jdk15to18:1.66
|    |    |    |    |    +--- org.jetbrains.kotlinx:kotlinx-coroutines-core-jvm:1.3.9
|    |    |    |    |    |    +--- org.jetbrains.kotlin:kotlin-stdlib:1.4.0 (*)
|    |    |    |    |    |    \--- org.jetbrains.kotlin:kotlin-stdlib-common:1.4.0
|    |    |    |    |    +--- com.squareup.retrofit2:retrofit:2.9.0 (*)
|    |    |    |    |    \--- com.squareup.retrofit2:converter-gson:2.9.0
|    |    |    |    |         +--- com.squareup.retrofit2:retrofit:2.9.0 (*)
|    |    |    |    |         \--- com.google.code.gson:gson:2.8.5
|    |    |    |    +--- org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.4.0 (*)
|    |    |    |    \--- org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.9 (*)

And I also google org.jetbrains.kotlinx:kotlinx-coroutines-core-jvm but found nothing related. For now, I try these to workaround class duplicated issue

    implementation('com.babylon.certificatetransparency:certificatetransparency-android:0.3.0') {
        exclude group: 'com.babylon.certificatetransparency', module: 'certificatetransparency'
    }

    implementation('com.babylon.certificatetransparency:certificatetransparency:0.3.0') {
        exclude group: 'org.jetbrains.kotlinx', module: 'kotlinx-coroutines-core-jvm'
    }

Enable explicitApi once https://youtrack.jetbrains.com/issue/KT-37652 is fixed

Hello I implemented this library with okhttp/certificate pinning (using TrustKit)
the problem is that i received an error even tho the hostname has the logs
checking in Google report
there is a CT's (screenshot)

im using the following config:

certificateTransparencyInterceptor {
        +"*.myhostname.io"
        failOnError = false
        logger = BasicAndroidCTLogger(BuildConfig.DEBUG)
    }

but I receive from logcat
CertificateTransparency: domain.myhostname.io Failure: No certificates
the okHttp is like this:

okHttpBuilder.addInterceptor(authInterceptor)
        .sslSocketFactory(OkHttp3Helper.getSSLSocketFactory(), OkHttp3Helper.getTrustManager())
        .addInterceptor(OkHttp3Helper.getPinningInterceptor())
        .addNetworkInterceptor(getCertificateTransparencyInterceptor())
        .sslSocketFactory(certificatePinning.getSSLSocketFactory(),
            certificatePinning.getTrustManager())
        .followRedirects(false)
        .followSslRedirects(false)
        .build()

im adding the intereptor in .addNetworkInterceptor
the certificate is logged by google_xenon2021and cloudflare_nimbus2021 and they are in the default list
https://www.gstatic.com/ct/log_list/log_list.json

I'm trying to figure out what the problem is but I have no idea,

Currently we can only specify hosts to process. The use case is this:

1: Configure a bunch of wild card domains to enable this for PLUS...
2: A bunch of overrides we want this disabled for.

Disabled hosts higher priority than enabled ones obviously.

The behaviour then allows our configuration to represent a bunch of exceptions that we need to work our way through and get certs in place that support SCTs.

Might look like this...

new CertificateTransparencyInterception.Builder()
.addHost(".example.com")
.addHost(".anotherexample.com")
.addIgnoreHost("domainwithoutscts.example.com")
.build()

Hopefully that makes sense. Over time our development team would look to remove the ignored hosts as they get fixed to contain certs containing the SCTs. Once all ignores gone we'd be free of operational burden (kinda) as the wild card hosts protect all our domains without having to keep specifying exact details.

Version -1.4
Error - Failure: Too few trusted SCTs, required 2, found 1 in {u9nfvB+KcbWTlCOXqpJ7RzhXlQqrUugakJZkNo4e0YU==Valid SCT, h3W/51l8+IxDmV+9827/Vo1HVjb/SrVgwbTq/16ggw8==SCT timestamp, 1573258714144, is greater than the log server validity, 1588550440.}

Hi,
I am trying to retrieve the certificate log entries from Ctlog servers using HttpLogClient from com.babylon.certificatetransparency:certificatetransparency:0.3.0 jar but I am getting Serialization exception as mentioned below while getting the parseLogEntry.

Exception in thread "main" com.babylon.certificatetransparency.internal.exceptions.SerializationException: Extra data corrupted.
at com.babylon.certificatetransparency.internal.serialization.Deserializer.parsePreCertificateChainEntry(Deserializer.kt:266)
at com.babylon.certificatetransparency.internal.serialization.Deserializer.parseLogEntry(Deserializer.kt:153)
at com.babylon.certificatetransparency.internal.logclient.model.network.GetEntriesResponse.toParsedLogEntries(GetEntriesResponse.kt:68)
at com.babylon.certificatetransparency.internal.logclient.HttpLogClient.getLogEntries(HttpLogClient.kt:116)
at com.qualys.certificate.transparency.TestCtLog.main(TestCtLog.java:24)

public class TestCtLog {
public static void main(String args[]) {

	try {
		HttpLogClient client = (HttpLogClient) LogClientFactory.INSTANCE
				.create("http://ct.googleapis.com/logs/argon2021/ct/v1/");
		// SignedTreeHead sth = client.getLogSTH();
		SignedTreeHead sth = client.getLogSth$certificatetransparency();
		List<com.babylon.certificatetransparency.internal.logclient.model.ParsedLogEntry> ent = client
				.getLogEntries(0, 10);

		com.babylon.certificatetransparency.internal.logclient.model.ParsedLogEntry logEntry = ent.get(0);
		LogEntry.X509ChainEntry entry = (X509ChainEntry) logEntry.getLogEntry();
		byte[] leafCert = entry.getLeafCertificate();

		X509Certificate cert = (X509Certificate) CertificateFactory.getInstance("X509")
				.generateCertificate(new ByteArrayInputStream(leafCert));

		System.out.println(cert);
	} catch (CertificateException e) {
		// TODO Auto-generated catch block
		e.printStackTrace();
	}

}

}

Hello,

Thank you for this project, you have done a great job by implementing certificate transparency for Android. My domain can be served securely on Android applications.

If I understand correctly the code, everything relies on the gstatic.com domain and its security: how is domain trusted? How is it protected against MITM?

Thank you

I am trying to use the this module in an java based Android SDK and it comes back with this error.

I did add the dependencies in the build.gradle, am i missing any other dependencies?

implementation 'com.babylon.certificatetransparency:certificatetransparency:0.1.2'
implementation "org.jetbrains.kotlin:kotlin-stdlib:1.3.31"

java.lang.NoClassDefFoundError: Failed resolution of: Lcom/babylon/certificatetransparency/internal/loglist/InMemoryDataSource;
        at com.babylon.certificatetransparency.internal.verifier.CertificateTransparencyBase.<init>(CertificateTransparencyBase.kt:58)
        at com.babylon.certificatetransparency.internal.verifier.CertificateTransparencyBase.<init>(CertificateTransparencyBase.kt:44)
        at com.babylon.certificatetransparency.internal.verifier.CertificateTransparencyInterceptor.<init>(CertificateTransparencyInterceptor.kt:37)
        at com.babylon.certificatetransparency.CTInterceptorBuilder.build(CTInterceptorBuilder.kt:144)

I use OKHttp in my project and wonder whether I could use ip instead domain (wildcards) as below ?

val interceptor = certificateTransparencyInterceptor {
    +"my_ip1",
    +"my_ip2"
}

val client = OkHttpClient.Builder().apply {
    addNetworkInterceptor(interceptor)
}.build()

Here's my configuration:

Android Studio 3.4.1
Build #AI-183.6156.11.34.5522156, built on May 1, 2019
JRE: 1.8.0_152-release-1343-b01 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
macOS 10.14.4

Admittedly, this issue could stem from a few different sources (R8, New Relic, and this lib), but I only noticed this issue when New Relic choked on it. Verified that it was only after adding the certificate-transparency-android lib that the error was thrown, so thought I'd start here.

06:28:58.833 [QUIET] [system.out] [newrelic.debug] [ClassTransformer] transformClassBytes: [module-info.class]
06:28:58.833 [QUIET] [system.out] [newrelic.error] Unfortunately, an error has occurred while processing an unknown class. Please copy your build logs and the jar containing this class and visit http://support.newrelic.com, thanks!
06:28:58.833 [QUIET] [system.out] null

Supposedly, the JPMS error was fixed in 3.4.0-beta, but it also depends on whether the library itself supports JPMS and/or was built with R8. Moreover, I did some research and found that module-info.class is only supported / used in Java 9+, which Android does not currently support.

Is this something that you could fix on your side?

Hi, first thank you for this library.

We recently learned that starting from May 1, 2021 JCenter will stop working.

Do you publish only to JCenter currently? We were unable to get this dependency from google() or mavenCentral().

Do you plan to publish all (including older) artifacts in the new place? We're currently on v0.3.0.

Hi,

I want to use the LogClientService to retrieve CT entries. Is that possible from outside of the project within a Java project?
I found no way so far.

Thanks!

Android Studio details:

Android Studio 3.4.1
Build #AI-183.6156.11.34.5522156, built on May 1, 2019
JRE: 1.8.0_152-release-1343-b01 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
macOS 10.14.4

Repro steps:

1. Add implementation 'com.babylon.certificatetransparency:certificatetransparency-android:0.1.2' to build.gradle dependencies.
2. Attempt to Sync Project with Gradle files
3. Note failure: ERROR: Failed to resolve: com.babylon.certificatetransparency:certificatetransparency-android:0.1.2

Additional Details:

• Regular Java dependency resolves just fine: implementation 'com.babylon.certificatetransparency:certificatetransparency:0.1.2'
• Previous version of -android (0.1.1) resolves without issue: implementation 'com.babylon.certificatetransparency:certificatetransparency-android:0.1.1
• Attempted to clean / invalidate and restart with no luck.
• Added the maven url (maven { url 'https://dl.bintray.com/babylonpartners/maven' }) to my repositories in the build.gradle file, which I hadn't needed to do before, and it still didn't resolve.

Let me know if there's something else I can try...

For example, the CertificateChainCleaner is in our internal package and we don't guarantee any API stability in that package across releases. For example, it'll break when you go from OkHttp 3.14 to 4.0.

You should either skip it (certificates from OkHttp are already cleaned) or copy the cleaner code into your own codebase!

Hi,

Since a few days, I encounter this error when gradle syncing my app project :

Could not resolve com.babylon.certificatetransparency:certificatetransparency-android:0.3.0.
> Could not get resource 'https://plugins.gradle.org/m2/com/babylon/certificatetransparency/certificatetransparency-android/0.3.0/certificatetransparency-android-0.3.0.pom'.
> Could not GET 'https://plugins.gradle.org/m2/com/babylon/certificatetransparency/certificatetransparency-android/0.3.0/certificatetransparency-android-0.3.0.pom'.
> Connect to www-cache-nrs.si.fr.intraorange:3128 [www-cache-nrs.si.fr.intraorange/10.242.190.3] failed: Connection timed out: connect

By following this url , I'm redirected to https://jcenter.bintray.com/com/babylon/certificatetransparency/certificatetransparency-android/0.3.0/certificatetransparency-android-0.3.0.pom with a 403 Forbidden error.

This url works in http and not https.

Do you have any solution to fix that problem ?

Thank you.

At some point a new v2 schema for the log list will be released.
https://groups.google.com/forum/?fromgroups=#!topic/certificate-transparency/zZwGExvQeiE

At such a time this library needs to update to use it.

After including the library I am unable to run the app, build is failing, I haven't used the library in code yet. Below is a screenshot -

We're seeing this crash stacktrace when we attempted to rollout 0.2.0 to our app, looks to be happening for about 6% of users across all Android OS versions. We stopped the rollout for now and removed the certificatetransparency dependency until this can be addressed. I haven't been able to reproduce this error, we're only seeing it in the wild.

Fatal Exception: java.util.NoSuchElementException
       at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:366)
       at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:416)
       at java.util.ServiceLoader$1.next(ServiceLoader.java:494)
       at kotlin.collections.CollectionsKt___CollectionsKt.firstOrNull(CollectionsKt___CollectionsKt.java:224)
       at kotlin.collections.CollectionsKt.firstOrNull(CollectionsKt.java)
       at com.babylon.certificatetransparency.chaincleaner.CertificateChainCleaner$Companion.get(CertificateChainCleaner.java:27)
       at com.babylon.certificatetransparency.internal.verifier.CertificateTransparencyBase$cleaner$2.invoke(CertificateTransparencyBase.java:63)
       at com.babylon.certificatetransparency.internal.verifier.CertificateTransparencyBase$cleaner$2.invoke(CertificateTransparencyBase.java:42)
       at kotlin.SynchronizedLazyImpl.getValue(SynchronizedLazyImpl.java:74)
       at com.babylon.certificatetransparency.internal.verifier.CertificateTransparencyBase.hasValidSignedCertificateTimestamp(CertificateTransparencyBase.java:96)
       at com.babylon.certificatetransparency.internal.verifier.CertificateTransparencyBase.verifyCertificateTransparency(CertificateTransparencyBase.java:76)
       at com.babylon.certificatetransparency.internal.verifier.CertificateTransparencyInterceptor.intercept(CertificateTransparencyInterceptor.java:51)
       at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:112)
       at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:37)

After compiling this project source code into jar and AAR library, we encountered a runtime error as shown in the figure below. How should we handle it? Thank you!

D/AndroidRuntime: Shutting down VM
E/AndroidRuntime: FATAL EXCEPTION: main
Process: pra.jyao20.ctl3, PID: 29195
java.lang.NoClassDefFoundError: Failed resolution of: Lretrofit2/Retrofit$Builder;
at com.babylon.certificatetransparency.internal.loglist.LogListDataSourceFactory.create(LogListDataSourceFactory.kt:34)
at com.babylon.certificatetransparency.internal.verifier.CertificateTransparencyBase.(CertificateTransparencyBase.kt:69)
at com.babylon.certificatetransparency.internal.verifier.CertificateTransparencyInterceptor.(CertificateTransparencyInterceptor.kt:44)
at com.babylon.certificatetransparency.CTInterceptorBuilder.build(CTInterceptorBuilder.kt:231)
at pra.jyao20.ctl3.MainActivity.addCt(MainActivity.kt:94)
at pra.jyao20.ctl3.MainActivity.okHttpRequest(MainActivity.kt:46)
at pra.jyao20.ctl3.MainActivity.access$okHttpRequest(MainActivity.kt:18)
at pra.jyao20.ctl3.MainActivity$onCreate$2.onClick(MainActivity.kt:32)
at android.view.View.performClick(View.java:7250)
at android.view.View.performClickInternal(View.java:7227)
at android.view.View.access$3500(View.java:819)
at android.view.View$PerformClick.run(View.java:27749)
at android.os.Handler.handleCallback(Handler.java:883)
at android.os.Handler.dispatchMessage(Handler.java:100)
at android.os.Looper.loop(Looper.java:224)
at android.app.ActivityThread.main(ActivityThread.java:7562)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:539)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:950)
Caused by: java.lang.ClassNotFoundException: Didn't find class "retrofit2.Retrofit$Builder" on path: DexPathList[[zip file "/data/app/pra.jyao20.ctl3-utW69wtJypMQlPcmdrdHkw==/base.apk"],nativeLibraryDirectories=[/data/app/pra.jyao20.ctl3-utW69wtJypMQlPcmdrdHkw==/lib/arm64, /system/lib64, /system/product/lib64]]
at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:230)
at java.lang.ClassLoader.loadClass(ClassLoader.java:379)
at java.lang.ClassLoader.loadClass(ClassLoader.java:312)
at com.babylon.certificatetransparency.internal.loglist.LogListDataSourceFactory.create(LogListDataSourceFactory.kt:34) 
at com.babylon.certificatetransparency.internal.verifier.CertificateTransparencyBase.(CertificateTransparencyBase.kt:69) 
at com.babylon.certificatetransparency.internal.verifier.CertificateTransparencyInterceptor.(CertificateTransparencyInterceptor.kt:44) 
at com.babylon.certificatetransparency.CTInterceptorBuilder.build(CTInterceptorBuilder.kt:231) 
at pra.jyao20.ctl3.MainActivity.addCt(MainActivity.kt:94) 
at pra.jyao20.ctl3.MainActivity.okHttpRequest(MainActivity.kt:46) 
at pra.jyao20.ctl3.MainActivity.access$okHttpRequest(MainActivity.kt:18) 
at pra.jyao20.ctl3.MainActivity$onCreate$2.onClick(MainActivity.kt:32) 
at android.view.View.performClick(View.java:7250) 
at android.view.View.performClickInternal(View.java:7227) 
at android.view.View.access$3500(View.java:819) 
at android.view.View$PerformClick.run(View.java:27749) 
at android.os.Handler.handleCallback(Handler.java:883) 
at android.os.Handler.dispatchMessage(Handler.java:100) 
at android.os.Looper.loop(Looper.java:224) 
at android.app.ActivityThread.main(ActivityThread.java:7562) 
at java.lang.reflect.Method.invoke(Native Method) 
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:539) 
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:950) 
E/MQSEventManagerDelegate: failed to get MQSService.
I/Process: Sending signal. PID: 29195 SIG: 9

Certificate Transparency: Success: SCT trusted logs {9lyUL9F3MCIUVBgIMJRWjuNNExkzv98MLyALzE7xZOM==Valid SCT, lCC8Ho7VjWyIcx+CiyIsDdHaTV5sT5Q9YdtOL1hNosI==Valid SCT}

Likely = at end of key makes the map toString() confusing. Not a big issue.

Enabling R8 causes crash

Fatal Exception: java.lang.RuntimeException: Unable to invoke no-args constructor for class c1.c.a.h.c.u.a.b. Registering an InstanceCreator with Gson for this type may fix this problem.
       at com.google.gson.internal.ConstructorConstructor$14.construct(ConstructorConstructor.java:32)
       at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.a(ReflectiveTypeAdapterFactory.java:15)
       at com.google.gson.Gson.fromJson(Gson.java:5)
       at com.google.gson.Gson.fromJson(Gson.java:14)
       at com.google.gson.Gson.fromJson(Gson.java:3)
       at com.babylon.certificatetransparency.internal.loglist.parser.LogListJsonParserV2.parseJson(LogListJsonParserV2.java:3)
       at com.babylon.certificatetransparency.internal.loglist.parser.RawLogListToLogListResultTransformer.transformSuccess(RawLogListToLogListResultTransformer.java:3)
       at com.babylon.certificatetransparency.internal.loglist.LogListDataSourceFactory$create$2.invoke(LogListDataSourceFactory.java:1)

Crash can be fixed by adding following to project's proguard-rules.pro file

-keep class com.babylon.certificatetransparency.internal.loglist.model.v2.** { *; }

Rather than an explicit whitelist, run for public sites (based on Expect-CT header?) and collect expect-ct and store from previous runs

#drivebyfeaturesuggestion

java.lang.NoClassDefFoundError: Failed resolution of: Lorg/bouncycastle/util/io/pem/PemReader;
at com.babylon.certificatetransparency.internal.utils.PublicKeyFactory.fromPemString(PublicKeyFactory.kt:47)
at com.babylon.certificatetransparency.internal.loglist.GooglePublicKeyKt.(GooglePublicKey.kt:21)
at com.babylon.certificatetransparency.internal.loglist.GooglePublicKeyKt.getGoogleLogListPublicKey(GooglePublicKey.kt:21)
at com.babylon.certificatetransparency.internal.loglist.parser.LogListVerifier.(LogListVerifier.kt:28)
at com.babylon.certificatetransparency.internal.loglist.parser.RawLogListToLogListResultTransformer.(RawLogListToLogListResultTransformer.kt:33)
at com.babylon.certificatetransparency.internal.loglist.LogListDataSourceFactory.create(LogListDataSourceFactory.kt:34)
at com.babylon.certificatetransparency.internal.verifier.CertificateTransparencyBase.(CertificateTransparencyBase.kt:66)
at com.babylon.certificatetransparency.internal.verifier.CertificateTransparencyHostnameVerifier.(CertificateTransparencyHostnameVerifier.kt:40)
at com.babylon.certificatetransparency.CTHostnameVerifierBuilder.build(CTHostnameVerifierBuilder.kt:214)
at com.ikea.network.NetworkCaller.execute(NetworkCaller.java:98)
at com.ikea.shared.config.service.ConfigService.fetchConfig(ConfigService.java:164)
at com.ikea.shared.config.service.ConfigService.performFetchConfigOperation(ConfigService.java:202)
at com.ikea.shared.config.service.ConfigService.access$000(ConfigService.java:43)
at com.ikea.shared.config.service.ConfigService$2.run(ConfigService.java:179)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
at java.lang.Thread.run(Thread.java:761)
Caused by: java.lang.ClassNotFoundException: Didn't find class "org.bouncycastle.util.io.pem.PemReader" on path: DexPathList[[zip file "/system/framework/com.google.android.maps.jar", zip file "/data/app/com.ikea.ksa.kompis-1/base.apk"],nativeLibraryDirectories=[/data/app/com.ikea.ksa.kompis-1/lib/arm, /data/app/com.ikea.ksa.kompis-1/base.apk!/lib/armeabi-v7a, /system/lib, /vendor/lib]]
at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:56)
at java.lang.ClassLoader.loadClass(ClassLoader.java:380)
at java.lang.ClassLoader.loadClass(ClassLoader.java:312)
at com.babylon.certificatetransparency.internal.utils.PublicKeyFactory.fromPemString(PublicKeyFactory.kt:47)
at com.babylon.certificatetransparency.internal.loglist.GooglePublicKeyKt.(GooglePublicKey.kt:21)
at com.babylon.certificatetransparency.internal.loglist.GooglePublicKeyKt.getGoogleLogListPublicKey(GooglePublicKey.kt:21)
at com.babylon.certificatetransparency.internal.loglist.parser.LogListVerifier.(LogListVerifier.kt:28)
at com.babylon.certificatetransparency.internal.loglist.parser.RawLogListToLogListResultTransformer.(RawLogListToLogListResultTransformer.kt:33)
at com.babylon.certificatetransparency.internal.loglist.LogListDataSourceFactory.create(LogListDataSourceFactory.kt:34)
at com.babylon.certificatetransparency.internal.verifier.CertificateTransparencyBase.(CertificateTransparencyBase.kt:66)
at com.babylon.certificatetransparency.internal.verifier.CertificateTransparencyHostnameVerifier.(CertificateTransparencyHostnameVerifier.kt:40)
at com.babylon.certificatetransparency.CTHostnameVerifierBuilder.build(CTHostnameVerifierBuilder.kt:214)
at com.ikea.network.NetworkCaller.execute(NetworkCaller.java:98)
at com.ikea.shared.config.service.ConfigService.fetchConfig(ConfigService.java:164)
at com.ikea.shared.config.service.ConfigService.performFetchConfigOperation(ConfigService.java:202)
at com.ikea.shared.config.service.ConfigService.access$000(ConfigService.java:43)
at com.ikea.shared.config.service.ConfigService$2.run(ConfigService.java:179)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
at java.lang.Thread.run(Thread.java:761)

We're seeing a few of these exceptions in Crashlytics in our app:

Caused by java.security.NoSuchAlgorithmException: SHA-256 MessageDigest not available
       at sun.security.jca.GetInstance.getInstance(GetInstance.java:159)
       at java.security.Security.getImpl(Security.java:628)
       at java.security.MessageDigest.getInstance(MessageDigest.java:197)
       at com.babylon.certificatetransparency.internal.utils.PublicKeyExtKt.sha256Hash(PublicKeyExtKt.java:26)
       at com.babylon.certificatetransparency.loglist.LogServer.<init>(LogServer.java:34)
       at com.babylon.certificatetransparency.internal.loglist.parser.LogListJsonParserV2.buildLogServerList(LogListJsonParserV2.java:67)
       at com.babylon.certificatetransparency.internal.loglist.parser.LogListJsonParserV2.parseJson(LogListJsonParserV2.java:41)
       at com.babylon.certificatetransparency.internal.loglist.parser.RawLogListToLogListResultTransformer.transformSuccess(RawLogListToLogListResultTransformer.java:51)
       at com.babylon.certificatetransparency.internal.loglist.parser.RawLogListToLogListResultTransformer.transform(RawLogListToLogListResultTransformer.java:35)
       at com.babylon.certificatetransparency.internal.loglist.LogListDataSourceFactory$create$2.invoke(LogListDataSourceFactory.java:49)
       at com.babylon.certificatetransparency.internal.loglist.LogListDataSourceFactory$create$2.invoke(LogListDataSourceFactory.java:29)
       at com.babylon.certificatetransparency.datasource.DataSource$oneWayTransform$1.get(DataSource.java:113)
       at com.babylon.certificatetransparency.datasource.DataSource$reuseInflight$1$get$2.invokeSuspend(DataSource.java:86)
       at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(BaseContinuationImpl.java:33)
       at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.java:56)
       at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.java:571)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.java:738)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.java:678)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.java:665)

It's been reported from different devices and different OS versions, but most devices are running Android 10.

We're using the CT library together with Retrofit and OkHttpClient.

Hi,
I am using babylon 0.2 version lib.
Steps to Repro :

1. Open the application using charles- > Certificate transparency failes
2. Remove the Charles from app -> Still certificate transparency is shown with the following error

javax.net.ssl.SSLPeerUnverifiedException: Certificate transparency failed
at com.babylon.certificatetransparency.internal.verifier.CertificateTransparencyInterceptor.intercept(CertificateTransparencyInterceptor.kt:59)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:37)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:87)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:82)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:87)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:84)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:71)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:87)

Expected Result: App should work if Charles was removed when app is running.
Note: Certificate transparency is succeeded when the app is killed and started again.

Our Android users are having an intermittent issue with the following stack trace:

We're using v0.3.0 of your lib. Please let me know what other info would be of interest to you.

Fatal Exception: java.lang.NullPointerException: Attempt to invoke virtual method 'int java.lang.Object.hashCode()' on a null object reference at libcore.reflect.TypeVariableImpl.hashCode(TypeVariableImpl.java:47) at java.util.HashMap.hash(HashMap.java:338) at java.util.HashMap.containsKey(HashMap.java:595) at java.util.HashSet.contains(HashSet.java:203) at com.google.gson.internal.$Gson$Types.resolve(.java:346) at com.google.gson.internal.$Gson$Types.resolve(.java:381) at com.google.gson.internal.$Gson$Types.resolve(.java:337) at com.google.gson.internal.$Gson$Types.getSupertype(.java:283) at com.google.gson.internal.$Gson$Types.getCollectionElementType(.java:302) at com.google.gson.internal.bind.CollectionTypeAdapterFactory.create(CollectionTypeAdapterFactory.java:52) at com.google.gson.Gson.getAdapter(Gson.java:458) at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.createBoundField(ReflectiveTypeAdapterFactory.java:117) at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.getBoundFields(ReflectiveTypeAdapterFactory.java:166) at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.create(ReflectiveTypeAdapterFactory.java:102) at com.google.gson.Gson.getAdapter(Gson.java:458) at com.google.gson.Gson.fromJson(Gson.java:926) at com.google.gson.Gson.fromJson(Gson.java:892) at com.google.gson.Gson.fromJson(Gson.java:841) at com.google.gson.Gson.fromJson(Gson.java:813) at com.babylon.certificatetransparency.internal.loglist.parser.LogListJsonParserV2.parseJson(LogListJsonParserV2.java:36) at com.babylon.certificatetransparency.internal.loglist.parser.RawLogListToLogListResultTransformer.transformSuccess(RawLogListToLogListResultTransformer.java:51) at com.babylon.certificatetransparency.internal.loglist.parser.RawLogListToLogListResultTransformer.transform(RawLogListToLogListResultTransformer.java:35) at com.babylon.certificatetransparency.internal.loglist.LogListDataSourceFactory$create$2.invoke(LogListDataSourceFactory.java:49) at com.babylon.certificatetransparency.internal.loglist.LogListDataSourceFactory$create$2.invoke(LogListDataSourceFactory.java:29) at com.babylon.certificatetransparency.datasource.DataSource$oneWayTransform$1.get(DataSource.java:113) at com.babylon.certificatetransparency.datasource.DataSource$oneWayTransform$1$get$1.invokeSuspend(DataSource.java:11) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(BaseContinuationImpl.java:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.java:241) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.java:594) at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.java:60) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.java:740)

I suspect this is an issue on our end but any help you can provide would be greatly appreciated.

I am seeing the log
"Success: SCT not enabled for insecure connection" for all the api in my app. I am sure that they are secure and have valid SCT. Looking at the code i see that there is a check

val host = chain.request().url().host()
        val certs = chain.connection()?.handshake()?.peerCertificates()?.map { it as X509Certificate } ?: emptyList()

        val result = if (chain.connection()?.socket() is SSLSocket) {
            verifyCertificateTransparency(host, certs)
        } else {
            VerificationResult.Success.InsecureConnection(host)
        }

and in my case its hitting the else part because the connection is null and I see that in RealCall class the connection is sent as null.

My code:

eventApiClient: OkHttpClient.Builder
 val interceptor = certificateTransparencyInterceptor {
            +"*.xyz.com"
            +"*.asd.xyz.com"
            logger = object : Logger {
                override fun log(host: String, result: VerificationResult) {
                    println("$host -> $result")
                }
            }
        }

eventApiClient.addInterceptor(interceptor)
return Retrofit.Builder()
                .baseUrl(url)
                .client(eventApiClient.build())
                .build()
                   

Can you please check and tell me whats going wrong here

I added implementation "com.babylon.certificatetransparency:certificatetransparency-android:0.3.0" as the dependency
and in the OkHttpClient I made sure to add new CTInterceptorBuilder().includeHost("*.example.com").build() as a network interceptor.

but this is the error that I see,

Caused by: java.lang.NoClassDefFoundError: Failed resolution of: Lcom/babylon/certificatetransparency/CTInterceptorBuilderExtKt;
        
     Caused by: java.lang.ClassNotFoundException: Didn't find class "com.babylon.certificatetransparency.CTInterceptorBuilderExtKt" on path: DexPathList[[zip file]]
        at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:207)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:379)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:312)
        	... 60 more

Babylon library version : 0.2.0

We are getting the following error for production users.

Failure: Unable to load log servers with log-list.json failed to load with java.net.SocketTimeoutException: SSL handshake timed out at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake(Native Method) at com.android.org.conscrypt.NativeSsl.doHandshake(NativeSsl.java:375) at com.android.org.conscrypt.ConscryptFileDescriptorSocket.startHandshake(ConscryptFileDescriptorSocket.java:224) at okhttp3.internal.connection.RealConnection.connectTls(:351) at okhttp3.internal.connection.RealConnection.establishProtocol(:310) at okhttp3.internal.connection.RealConnection.connect(:178) at okhttp3.internal.connection.ExchangeFinder.findConnection(:236) at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(:109) at okhttp3.internal.connection.ExchangeFinder.find(:77) at okhttp3.internal.connection.Transmitter.newExchange$okhttp(:162) at okhttp3.internal.connection.ConnectInterceptor.intercept(:35) at okhttp3.internal.http.RealInterceptorChain.proceed(:112) at okhttp3.internal.http.RealInterceptorChain.proceed(:87) at okhttp3.internal.cache.CacheInterceptor.intercept(:82) at okhttp3.internal.http.RealInterceptorChain.proceed(:112) at okhttp3.internal.http.RealInterceptorChain.proceed(:87) at okhttp3.internal.http.BridgeInterceptor.intercept(:84) at okhttp3.internal.http.RealInterceptorChain.proceed(:112) at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(:71) at okhttp3.internal.http.RealInterceptorChain.proceed(:112) at okhttp3.internal.http.RealInterceptorChain.proceed(:87) at okhttp3.RealCall.getResponseWithInterceptorChain(:184) at okhttp3.RealCall$AsyncCall.run(:136) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.lang.Thread.run(Thread.java:764)

Could you please check this issue?

Research into server-side adoption of certificate transparency shows that X.509v3 extensions are currently the most popular way to retrieve SCTs, however to adhere to the Certificate Transparency v2 spec the library needs to add support for retrieving these over TLS extensions and OCSP stapling.

The same research shows how this can be implemented in pure Java with no native libraries.

See Server-side Adoption of Certificate Transparency

Sun setting on JCenter on March 31st 2021. The distribution of the library will have to be down via Maven Central.

https://jfrog.com/blog/into-the-sunset-bintray-jcenter-gocenter-and-chartcenter/

Started getting Invalid Signature for the same domain which was working fine until yesterday.

Seems like version 1 log_list stops working https://www.gstatic.com/ct/log_list/log_list.json

When I tried with version 2 log_list URL it is working fine as expected https://www.gstatic.com/ct/log_list/v2/log_list.json

@mattmook Please help me out

Hey @mattmook,

First of all kudos for the amazing work that you and the team have done!

While intercepting traffic using a proxy server(Charles, Burp Suite, etc.) we install the cert manually to the device trust store. If the certificate is added properly(overriding the default NSC configuration) we should be able to intercept the traffic in spite of CT checks in place based on the Mozilla docs there would be no enforcement. Can you please look into this?

Root CAs manually added to the trust store override and suppress Expect-CT reports/enforcement.
Reference: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Expect-CT

However, you mentioned that the certificate-transparency-android would be blocking the requests:

If when you run your app, certificate transparency causes a refused connection then you are all set. In this scenario, you also shouldn’t see your requests in the mitmproxy console window
Ref: https://medium.com/babylon-engineering/android-security-certificate-transparency-601c18157c44

Does this mean that you are explicitly blocking the requests in the case the CT checks fail which would be the case when we would be using the proxy server for traffic interception as the cert that the proxy would present would have the CNAME but won't have the SCTs for it?

Why we have to give each domain as +.domain.com.
Is there any restriction for enabling for all domains by default and restricting whichever does not have CT by adding -.domain.com

We found this library awesome and we wanted to contribute, but before that, we want to test our changes in our project.

We tried building the project just like you do on circle ci, using ./gradlew build.

It generated the aar for the android module, but the lib module didn't come as transitive dependency, making its files not accessible when consuming the generated aar.

Can you help us build the project? Also, it would be cool having this in the CONTRIBUTING.MD

Does the CT work if connected to charles proxy. We use this for debugging and tracking the api calls. When I am connected to Chales, I see that CT is failing with the logs
at com.babylon.certificatetransparency.chaincleaner.BasicCertificateChainCleaner.clean(BasicCertificateChainCleaner.kt:81)
at com.babylon.certificatetransparency.internal.verifier.CertificateTransparencyBase.verifyCertificateTransparency(CertificateTransparencyBase.kt:71)
at com.babylon.certificatetransparency.internal.verifier.CertificateTransparencyInterceptor.intercept(CertificateTransparencyInterceptor.kt:44)

Is this the expected behavior? Is there a way to get around it, so that we can have the CT enabled even when connected to charles.

I have performed the CT in the application and it is working as expected, But now I am facing problems with the certificate revocation checking. Is there any way I can perform revocation checking in Android?

2019-06-08 11:30:02.028 3260-3741/com.okhttpandroidapp W/AndroidNetworkManager: ct: api.twitter.com Failure: No certificates

http vs https seems like a separate concern, e.g. HSTS. Should the default be to allow http traffic still and skip the checks?

private void addCt(OkHttpClient.Builder okHttpBuilder, Set hosts ){
Logger defaultLogger = new Logger() {
@OverRide
public void log(@NotNull String s, @NotNull VerificationResult verificationResult) {
LogUtil.w("--- verify s %s, result: "+verificationResult, s);
}
};
CTInterceptorBuilder builder = new CTInterceptorBuilder()
.setFailOnError(true)
.setLogger(defaultLogger)
;
for (String host : hosts) {
builder.addHost(host);
}
Interceptor networkInterceptor = builder.build();
okHttpBuilder.addNetworkInterceptor(networkInterceptor);
}

Why the log told me:
result: Success: SCT not enabled xxx

Thank you!

Null pointer exception is thrown when trying to build the app with minifyEnabled = true while using your library.
I am using
implementation 'com.babylon.certificatetransparency:certificatetransparency-android:0.2.0'.

Could you please help me solve this issue?