Insecure TLS Certificate Checking in Android Apps

Since launching Guardsquare’s mobile application security testing tool AppSweep at the beginning of August, we have been monitoring the types of security issues found in Android apps. One of the predominant issue classes we saw were apps containing code that led to an insecure configuration of HTTPS connections.

This post will explain the technical background of why these configurations are insecure, and how attackers can take advantage of these situations. If you would like to learn more about how to properly secure HTTPS connections in Android apps, keep an eye out for the follow up post.

HTTPS-Related Findings in AppSweep

With HTTPS communication, the actual traffic is encrypted with the Transport Layer Security (TLS) protocol, which bases its trust model on the certificate presented by the web server. In order for the connection to be secure, the client needs to properly examine this server certificate and determine whether it is trustworthy.

After several months of Android application scans, we’ve noticed several common issues with certificate verification in more than 33% of all scanned builds:

1. We discovered that 9% of all builds submitted to AppSweep instructed WebView objects to ignore any TLS-related errors during connection establishment. This is insecure, as TLS errors are usually caused by invalid certificates, so ignoring them ultimately undermines the entire certificate checking process.
   
   
2. 25% of the apps scanned have a malfunctioning custom implementation of the X509TrustManager class. The X509TrustManager class is responsible for the validation of TLS certificate chains, so if its implementation is flawed, the entire security concept of the protocol is disabled.
   
   
3. Finally, 15% of the scanned apps were configured so that the general authenticity of the certificate was verified, but the last step of verifying the validity of the certificate’s host name was skipped. With this configuration, an attacker could present a valid certificate for malicious.com, even though the user was trying to connect to google.com.

Causes of Insecure TLS Configurations

As our scan results indicate, developers frequently override the secure default implementations for validating TLS certificates provided by the Android framework. A very prominent concept in the field of cryptography and IT security is to rely on well-known and tested existing algorithms and libraries, and to not “roll your own” crypto. This naturally applies to validating TLS certificates correctly as well.

This raises the question of why developers want to override the defaults anyway, and risk ending up with insecure implementations. Analyzing StackOverflow questions and publicly available blog posts gave us some insight into the reasons, which seem twofold:

1. Some apps need to establish secure connections to servers that use certificates issued by custom certificate authorities (CAs)
2. OWASP strongly recommends using certificate pinning to secure communication with backend servers in accordance with security best practices, but implementing this yourself is a potentially error-prone task

We will now dive deeper into these two reasons, exploring how they might result in insecure implementations and why developers include them in their apps.

Custom Certificate Authorities

Many developers want to create an app that has to communicate with a backend server using a certificate issued by a custom certificate authority. Certificate authorities (CAs) are a fundamental part of the TLS certificate system: By default, browsers and operating systems only trust connections to servers whose certificates have been issued by a well-known CA. In our example case, where the certificate was issued by an unknown custom CA, the connection attempt is aborted with an exception.

Developers often consult sites like StackOverflow when dealing with exceptions they do not know how to deal with. But these posts often get answered with workarounds that introduce security issues such as disabling the default SSL certificate checking mechanisms in part or even completely. Unfortunately, many developers use such solutions without thoroughly checking their validity, resulting in the use of vulnerable coding patterns that lead to the AppSweep findings mentioned above.

Certificate Pinning

Another reason why developers might resort to custom TLS configurations actually stems from the opposite desire. Malicious actors could add an attacker-controlled certificate authority to the default trust store, which can then be used to issue forged certificates for any server. You can protect your app from such a scenario by using certificate pinning. If you are interested in learning more about this technique, check out our prior blog post on SSL certificate pinning and its limitations.

Certificate pinning is good practice from a security point of view, but unfortunately many developers try to implement this pinning code themselves. This is a potentially error-prone task, resulting in modifications to the default certificate checking implementation that might ultimately make the result less secure than before.

Demonstrating the Risk of Broken TLS Handling

We now know about three frequently occurring types of incorrect TLS handling in Android applications, and we understand the developer's motivation for custom implementations. But we still haven’t discussed the actual risk that such a broken implementation introduces for the end-user and the developer.

To dive deeper into this, we’ve constructed a containerized demo environment enabling us to easily mimic the different scenarios and demonstrate their consequences in practice. On our GitHub repository, you can find all the files you need to execute the demos, for a hands-on experience while following the next sections.

In the next two sections, we’ll briefly describe our demo setup, after which we will use it to demonstrate three real life scenarios.

Demo Environment Setup

Our environment has 3 containers as shown in Figure 1, mimicking the client (mobile app), back-end & malicious entity.

• The first container in the setup is a web server that offers an HTML web page when clients connect to it. This data that is transferred to the user is considered sensitive, and thus offered via HTTPS. The catch is that it uses a certificate issued by a custom certificate authority, mimicking the scenario we’ve observed in the previously mentioned application scans.
• The second container contains an Android emulator running a sample app that wants to retrieve the data offered by the web server and display it in a WebView. In order to establish a connection despite the custom server certificate, it showcases different custom certificate handling implementations in each tab.
• The third container simulates a malicious actor that happens to be on the same network as the Android device. This situation may arise e.g. in public WiFi hotspots that can frequently be found in cafes and other public places.

Figure 1: Demo environment setup

Intercepting HTTPS Traffic

The final part of the setup is actually having the malicious container intercept the secure channel between the client and the server, and optionally modifying its content. To achieve this we’ll use two techniques sequentially:

• ARP Spoofing: This makes the Android emulator believe that the malicious container is in fact the web server and analogously tells the web server that it is the Android emulator. By doing so, all network packets that are to be exchanged between the two other containers are delivered to the malicious container instead.

Figure 2: ARP spoofing

• Proxying: Once the packets are being re-routed, the attacker becomes a man-in-the-middle (MITM) entity that takes care of forwarding the packets to the correct recipient. Like this, they have full control over the communication without any of the two other parties noticing the presence of a proxy.

Figure 3: Attacker acting like a proxy

The following three experiments showcase how this situation will look in practice, each showing a different one of the previously mentioned vulnerable HTTPS configurations in Android apps.

WebView Ignores All SSL Errors

The WebView widget is a popular choice for situations where HTML data received from a web server should be displayed to the user. The code snippet below shows how the WebView’sWebViewClientimplementation is substituted by a custom client that modifies the callback responsible for SSL error handling. This method is invoked when Android doesn’t trust the TLS certificates provided by the target server, giving the implementer the choice to eitherproceed()with the connection orcancel()it.

Thus, when your app needs to deal with a certificate that is untrusted by the operating system, overriding this error handler allows deviating from the default behavior. Ignoring the error and proceeding with every connection is a frequently proposed quick fix suggested in various places, such as the previously linked StackOverflow thread. As a result, the entire certificate verification process is nullified.

binding.webview.webViewClient = object : WebViewClient() {
    override fun onReceivedSslError(
        view: WebView?,
        handler: SslErrorHandler?,
        error: SslError?
    ) {
        handler?.proceed()
    }
}

Now let’s have a look at the video below: When the MITM proxy is activated by running thestart.shscript, you can see that further HTTPS requests suddenly result in a very different text being displayed in the WebView. This is a result of the traffic manipulation executed by the malicious actor. The user has no way of finding out about the dangerous situation. The transmitted password is intercepted by the MITM entity and they can use it however they like, while the user receives the modified password without any idea that it was modified.

Malfunctioning X509TrustManager Implementations

If any communication with a server should happen securely over HTTPS without a WebView, the usual way of establishing the connection is using theHttpsURLConnectionclass. Similar to the WebView connection client, this approach also verifies the server’s TLS certificate based on the certificate authorities known to Android. In our case, the certificate would not be accepted by default, as it has been issued by a custom certificate authority. This results in an SSLException being thrown when attempting to start the connection.

Thus, similarly to how theWebViewClientimplementation can be overridden, we can instruct theHttpsURLConnectionto use a customX509TrustManagerimplementation that is responsible for verifying the TLS certificate trustworthiness:

val insecureTrustManager = object : X509TrustManager {
    override fun checkClientTrusted(
        chain: Array?,
        authType: String?
    ) {
        // do nothing
    }

    override fun checkServerTrusted(
        chain: Array?,
        authType: String?
    ) {
        // do nothing
    }

    override fun getAcceptedIssuers() = null
}
val context = SSLContext.getInstance("TLSv1.3")
context.init(null, arrayOf(insecureTrustManager), SecureRandom())

val url = URL("https://www.mitmtest.com")
val connection = url.openConnection() as HttpsURLConnection
connection.sslSocketFactory = context.socketFactory

The easiest but most insecure solution to get rid of the SSLException, often proposed publicly, is to use the implementation shown above, which simply does nothing. This is equivalent to deeming every server as trustworthy. The correct behavior would be to throw an exception in case an invalid certificate is presented. But if the app doesn’t do this, the actual certificate checking gets neglected. Attackers can then freely intercept and modify the transferred data without any problems, as shown in the video. Of course, you can also create a secure trust manager implementation that accepts certificates issued by a custom CA, but this is a topic we will explore in the next blog post.

Disabled Host Name Checks

Another way TLS certificate checking can be made insecure is if the host name verification process is skipped. In this case, the certificate validation process is not fully disabled like in the previous case. In fact, any certificate that was issued by an untrusted certificate authority will be rejected.

However, there is another crucial step when validating a certificate: Not only does the certificate have to be signed by a trusted CA, it also needs to be issued specifically for use with the exact URL that is currently being accessed. If this step is skipped, a malicious actor can use their legitimately received certificate for “malicious.com” to pretend they are actually “google.com”. There should never be any need to do so, but a common way of turning off the host name verification is shown in the snippet below:

val url = URL("https://wrong.host.badssl.com")
val conn = url.openConnection() as HttpsURLConnection

// Create HostnameVerifier that accepts everything
conn.hostnameVerifier = HostnameVerifier { hostname, session -> true }

As shown in the video below, when a website is visited whose TLS certificate has not been issued for the target host name, the above configuration still establishes a connection without any warnings or exceptions. If you try connecting to https://wrong.host.badssl.com using your own browser, you will see that the connection is refused exactly because the host name verification failed (e.g. in Google Chrome, this is called ERR_CERT_COMMON_NAME_INVALID).

Avoiding Vulnerabilities in Network-Facing Android Apps

When your app needs to communicate with a backend server, you should always use encrypted and authenticated channels like TLS in order to protect your users from eavesdropping and manipulation of traffic. It is best practice to use well-established procedures and libraries instead of custom implementations in IT security to minimize the risk of security issues. That’s why it’s safer to use well-known TLS libraries, and leverage generally trusted certificate authorities.

Using custom certificate authorities should be avoided as much as possible. They should only be used as a last resort in your security infrastructure, since using them makes the certificate verification process more error-prone. When you try to force Android to accept your custom certificates and establish a connection to the server, several things can go wrong.

For example, implementation errors could include skipping parts of the required certificate checking process or using workarounds for runtime exceptions by disabling certificate checks completely. This can allow attackers to intercept the encrypted traffic without the user noticing, as we have seen in the examples above.

But if using a custom certificate is the only option you have, your main focus should not be to establish the connection at any cost. Instead, you should focus on enforcing strict verification of the chain of trust. Avoiding crashes due to SSLExceptions by simply skipping all certificate checks is a very dangerous approach.

We hope that our findings and hands-on examples helped increase awareness for the need for secure TLS certificate checking implementations. If you are curious about properly making your Android app connect to servers using certificates by custom certificate authorities, then make sure to check out our upcoming blog post.