Frida for Scrapers: SSL Unpinning Android Apps to Extract API Data


The Scraper
Bypass Methods
Mobile apps often talk to the same backend as the web interface — but without the anti-bot layers guarding the web frontend. The app's API endpoints are usually cleaner, more structured, and less protected than what you'd scrape from HTML.
The barrier is SSL pinning: many apps verify that the certificate they receive matches a known-good certificate embedded in the app, preventing machine-in-the-middle inspection. Frida removes that barrier.
This guide covers SSL unpinning an Android app to map its API surface for scraping.
Prerequisites
Android device (rooted) or Android emulator
Python 3.11+
adb(Android Debug Bridge) installed and in PATHFrida server running on the device
mitmproxyfor traffic inspectionTarget APK installed on the device
Legal note: Only perform this on apps you have authorization to test, or on your own apps in a development environment. This guide is for security research and legitimate data collection from your own authorized targets.
Step 1: Set Up the Android Environment
Using an Emulator (Recommended for Getting Started)
Android Studio's emulator is the easiest path. Use a Google Play-free image (AOSP) for root access:
# List available system images sdkmanager --list | grep "system-images" # Create an emulator with a rootable image avdmanager create avd \ -n scraper_device \ -k "system-images;android-33;google_apis;x86_64" \ -d "pixel_6" # Start with writable system partition emulator -avd scraper_device -writable-system
Install Frida Server on the Device
# Find device architecture adb shell getprop ro.product.cpu.abi # → x86_64 # Download matching frida-server from github.com/frida/frida/releases # Example: frida-server-16.x.x-android-x86_64.xz xz -d frida-server-16.x.x-android-x86_64.xz adb push frida-server-16.x.x-android-x86_64 /data/local/tmp/frida-server adb shell chmod 755 /data/local/tmp/frida-server # Start frida-server (run as root) adb shell su -c "/data/local/tmp/frida-server &"
Verify it's working:
frida-ps -U # Should list running processes on the device
Step 2: Set Up mitmproxy
# Start mitmproxy mitmproxy --listen-port 8080 # Or mitmdump for headless/scripted inspection mitmdump -p 8080 -s log_requests.py
Install the mitmproxy CA certificate on the device:
# Route device traffic through your machine's mitmproxy adb shell settings put global http_proxy 192.168.x.x:8080 # Your machine's IP # Install mitmproxy cert # Navigate to http://mitm.it in the device browser # Download and install the Android certificate
At this point, HTTP apps work. HTTPS apps with SSL pinning will fail with a certificate error.
Step 3: SSL Unpinning with Frida
The Universal Unpinning Script
The most reliable starting point is the community-maintained universal unpinning script from Frida CodeShare. It hooks common SSL pinning implementations across most apps:
# Run the universal unpinning script against a target app frida -U \ -l https://codeshare.frida.re/@pcipolloni/universal-android-ssl-pinning-bypass-with-frida/ \ -f com.target.app
This hooks OkHttp3, TrustKit, Conscrypt, and the standard X509TrustManager implementation — covering the vast majority of Android apps.
A Manual Unpinning Script (When Universal Fails)
For apps with custom pinning implementations, you need to hook the specific pinning logic. First, identify it:
# Find pinning-related method names in the APK apktool d target-app.apk -o target_decoded grep -r "CertificatePinner\|TrustManager\|checkServerTrusted\|pinnedCerts" target_decoded
Then write a targeted hook:
// custom_unpin.js Java.perform(function() { console.log('[*] Starting SSL unpinning'); // Hook OkHttp3 CertificatePinner try { var CertificatePinner = Java.use('okhttp3.CertificatePinner'); CertificatePinner.check.overload('java.lang.String', 'java.util.List') .implementation = function(hostname, peerCertificates) { console.log('[+] OkHttp3 CertificatePinner.check bypassed for: ' + hostname); return; // Skip the check }; } catch(e) { console.log('[-] OkHttp3 not found: ' + e); } // Hook TrustManager try { var X509TrustManager = Java.use('javax.net.ssl.X509TrustManager'); var TrustManagerImpl = Java.use('com.android.org.conscrypt.TrustManagerImpl'); TrustManagerImpl.verifyChain.implementation = function( untrustedChain, trustAnchorChain, host, clientAuth, ocspData, tlsSctData ) { console.log('[+] TrustManagerImpl.verifyChain bypassed for: ' + host); return untrustedChain; }; } catch(e) { console.log('[-] TrustManagerImpl hook failed: ' + e); } console.log('[*] SSL unpinning complete'); });
Step 4: Capturing and Analyzing Traffic
Once unpinning is active, all HTTPS traffic from the app flows through mitmproxy in plaintext. Use mitmdump with a script to log and filter requests:
# log_requests.py — mitmproxy addon script import json from mitmproxy import http class APILogger: def __init__(self): self.api_calls = [] def request(self, flow: http.HTTPFlow) -> None: # Filter to API calls only if 'api.' in flow.request.host or '/v1/' in flow.request.path or '/v2/' in flow.request.path: call = { 'method': flow.request.method, 'url': flow.request.pretty_url, 'headers': dict(flow.request.headers), 'body': flow.request.text if flow.request.text else None, } self.api_calls.append(call) print(f"[API] {call['method']} {call['url']}") def response(self, flow: http.HTTPFlow) -> None: if 'api.' in flow.request.host: try: body = json.loads(flow.response.text) print(f"[RESPONSE] {json.dumps(body, indent=2)[:500]}") except: pass addons = [APILogger()]
Interact with the app manually, triggering the data you want to collect. You'll see every API call in the terminal.
Step 5: Replicating the API Calls
Once you've mapped the API surface, reproduce the calls in Python:
import httpx import json # Headers extracted from mitmproxy traffic HEADERS = { 'User-Agent': 'TargetApp/4.2.1 (Android 13; Pixel 6)', 'Authorization': 'Bearer YOUR_TOKEN_FROM_CAPTURE', 'X-App-Version': '4.2.1', 'X-Device-ID': 'extracted-device-id', 'Accept': 'application/json', 'Content-Type': 'application/json', } # Evomi mobile proxy for matching traffic profile PROXY = 'http://USERNAME:PASSWORD@mobile.evomi.com:2000' async def call_api(endpoint: str, params: dict) -> dict: async with httpx.AsyncClient( proxies={'http://': PROXY, 'https://': PROXY}, headers=HEADERS, timeout=30.0, ) as client: response = await client.get(endpoint, params=params) response.raise_for_status() return response.json() # Example: replicate a product search call async def search_products(query: str, location: str) -> list: data = await call_api( 'https://api.target-app.com/v2/products/search', {'q': query, 'location': location, 'limit': 50} ) return data.get('results', [])
Using Evomi's mobile proxies here matches the expected IP profile — real 4G/5G carrier IPs that mobile API backends treat as genuine app traffic, not datacenter or even residential IPs.
Common Pitfalls
Pitfall 1: Certificate not trusted at OS level. Android 7+ doesn't trust user-installed certificates for app traffic by default. The mitmproxy certificate needs to be installed as a system-level CA. On rooted devices:
# Get cert hash openssl x509 -inform PEM -subject_hash_old -in ~/.mitmproxy/mitmproxy-ca-cert.pem | head -1 # → e8d04e88 adb remount # Requires rooted device adb push ~/.mitmproxy/mitmproxy-ca-cert.pem /system/etc/security/cacerts/e8d04e88.0 adb shell chmod 644 /system/etc/security/cacerts/e8d04e88.0
Pitfall 2: App detects Frida. Some apps check for Frida's presence using process name scanning or port detection. Use -l with an anti-detection script to obfuscate Frida's presence, or try objection which includes built-in anti-detection.
Pitfall 3: Token expiry. Bearer tokens extracted during mitmproxy capture expire. Build token refresh logic into your scraper based on how the app authenticates (usually a login endpoint that returns a fresh token).
Conclusion
Mobile API extraction is a powerful technique precisely because it bypasses the browser-layer defenses that make web scraping expensive. Once you've mapped an app's API surface with Frida and mitmproxy, the actual data collection is often simpler and more reliable than browser automation.
Pair it with Evomi's mobile proxies for traffic that matches the expected source profile, and you have a complete mobile extraction stack. Test it free with Evomi's trial before committing to a volume plan.
Mobile apps often talk to the same backend as the web interface — but without the anti-bot layers guarding the web frontend. The app's API endpoints are usually cleaner, more structured, and less protected than what you'd scrape from HTML.
The barrier is SSL pinning: many apps verify that the certificate they receive matches a known-good certificate embedded in the app, preventing machine-in-the-middle inspection. Frida removes that barrier.
This guide covers SSL unpinning an Android app to map its API surface for scraping.
Prerequisites
Android device (rooted) or Android emulator
Python 3.11+
adb(Android Debug Bridge) installed and in PATHFrida server running on the device
mitmproxyfor traffic inspectionTarget APK installed on the device
Legal note: Only perform this on apps you have authorization to test, or on your own apps in a development environment. This guide is for security research and legitimate data collection from your own authorized targets.
Step 1: Set Up the Android Environment
Using an Emulator (Recommended for Getting Started)
Android Studio's emulator is the easiest path. Use a Google Play-free image (AOSP) for root access:
# List available system images sdkmanager --list | grep "system-images" # Create an emulator with a rootable image avdmanager create avd \ -n scraper_device \ -k "system-images;android-33;google_apis;x86_64" \ -d "pixel_6" # Start with writable system partition emulator -avd scraper_device -writable-system
Install Frida Server on the Device
# Find device architecture adb shell getprop ro.product.cpu.abi # → x86_64 # Download matching frida-server from github.com/frida/frida/releases # Example: frida-server-16.x.x-android-x86_64.xz xz -d frida-server-16.x.x-android-x86_64.xz adb push frida-server-16.x.x-android-x86_64 /data/local/tmp/frida-server adb shell chmod 755 /data/local/tmp/frida-server # Start frida-server (run as root) adb shell su -c "/data/local/tmp/frida-server &"
Verify it's working:
frida-ps -U # Should list running processes on the device
Step 2: Set Up mitmproxy
# Start mitmproxy mitmproxy --listen-port 8080 # Or mitmdump for headless/scripted inspection mitmdump -p 8080 -s log_requests.py
Install the mitmproxy CA certificate on the device:
# Route device traffic through your machine's mitmproxy adb shell settings put global http_proxy 192.168.x.x:8080 # Your machine's IP # Install mitmproxy cert # Navigate to http://mitm.it in the device browser # Download and install the Android certificate
At this point, HTTP apps work. HTTPS apps with SSL pinning will fail with a certificate error.
Step 3: SSL Unpinning with Frida
The Universal Unpinning Script
The most reliable starting point is the community-maintained universal unpinning script from Frida CodeShare. It hooks common SSL pinning implementations across most apps:
# Run the universal unpinning script against a target app frida -U \ -l https://codeshare.frida.re/@pcipolloni/universal-android-ssl-pinning-bypass-with-frida/ \ -f com.target.app
This hooks OkHttp3, TrustKit, Conscrypt, and the standard X509TrustManager implementation — covering the vast majority of Android apps.
A Manual Unpinning Script (When Universal Fails)
For apps with custom pinning implementations, you need to hook the specific pinning logic. First, identify it:
# Find pinning-related method names in the APK apktool d target-app.apk -o target_decoded grep -r "CertificatePinner\|TrustManager\|checkServerTrusted\|pinnedCerts" target_decoded
Then write a targeted hook:
// custom_unpin.js Java.perform(function() { console.log('[*] Starting SSL unpinning'); // Hook OkHttp3 CertificatePinner try { var CertificatePinner = Java.use('okhttp3.CertificatePinner'); CertificatePinner.check.overload('java.lang.String', 'java.util.List') .implementation = function(hostname, peerCertificates) { console.log('[+] OkHttp3 CertificatePinner.check bypassed for: ' + hostname); return; // Skip the check }; } catch(e) { console.log('[-] OkHttp3 not found: ' + e); } // Hook TrustManager try { var X509TrustManager = Java.use('javax.net.ssl.X509TrustManager'); var TrustManagerImpl = Java.use('com.android.org.conscrypt.TrustManagerImpl'); TrustManagerImpl.verifyChain.implementation = function( untrustedChain, trustAnchorChain, host, clientAuth, ocspData, tlsSctData ) { console.log('[+] TrustManagerImpl.verifyChain bypassed for: ' + host); return untrustedChain; }; } catch(e) { console.log('[-] TrustManagerImpl hook failed: ' + e); } console.log('[*] SSL unpinning complete'); });
Step 4: Capturing and Analyzing Traffic
Once unpinning is active, all HTTPS traffic from the app flows through mitmproxy in plaintext. Use mitmdump with a script to log and filter requests:
# log_requests.py — mitmproxy addon script import json from mitmproxy import http class APILogger: def __init__(self): self.api_calls = [] def request(self, flow: http.HTTPFlow) -> None: # Filter to API calls only if 'api.' in flow.request.host or '/v1/' in flow.request.path or '/v2/' in flow.request.path: call = { 'method': flow.request.method, 'url': flow.request.pretty_url, 'headers': dict(flow.request.headers), 'body': flow.request.text if flow.request.text else None, } self.api_calls.append(call) print(f"[API] {call['method']} {call['url']}") def response(self, flow: http.HTTPFlow) -> None: if 'api.' in flow.request.host: try: body = json.loads(flow.response.text) print(f"[RESPONSE] {json.dumps(body, indent=2)[:500]}") except: pass addons = [APILogger()]
Interact with the app manually, triggering the data you want to collect. You'll see every API call in the terminal.
Step 5: Replicating the API Calls
Once you've mapped the API surface, reproduce the calls in Python:
import httpx import json # Headers extracted from mitmproxy traffic HEADERS = { 'User-Agent': 'TargetApp/4.2.1 (Android 13; Pixel 6)', 'Authorization': 'Bearer YOUR_TOKEN_FROM_CAPTURE', 'X-App-Version': '4.2.1', 'X-Device-ID': 'extracted-device-id', 'Accept': 'application/json', 'Content-Type': 'application/json', } # Evomi mobile proxy for matching traffic profile PROXY = 'http://USERNAME:PASSWORD@mobile.evomi.com:2000' async def call_api(endpoint: str, params: dict) -> dict: async with httpx.AsyncClient( proxies={'http://': PROXY, 'https://': PROXY}, headers=HEADERS, timeout=30.0, ) as client: response = await client.get(endpoint, params=params) response.raise_for_status() return response.json() # Example: replicate a product search call async def search_products(query: str, location: str) -> list: data = await call_api( 'https://api.target-app.com/v2/products/search', {'q': query, 'location': location, 'limit': 50} ) return data.get('results', [])
Using Evomi's mobile proxies here matches the expected IP profile — real 4G/5G carrier IPs that mobile API backends treat as genuine app traffic, not datacenter or even residential IPs.
Common Pitfalls
Pitfall 1: Certificate not trusted at OS level. Android 7+ doesn't trust user-installed certificates for app traffic by default. The mitmproxy certificate needs to be installed as a system-level CA. On rooted devices:
# Get cert hash openssl x509 -inform PEM -subject_hash_old -in ~/.mitmproxy/mitmproxy-ca-cert.pem | head -1 # → e8d04e88 adb remount # Requires rooted device adb push ~/.mitmproxy/mitmproxy-ca-cert.pem /system/etc/security/cacerts/e8d04e88.0 adb shell chmod 644 /system/etc/security/cacerts/e8d04e88.0
Pitfall 2: App detects Frida. Some apps check for Frida's presence using process name scanning or port detection. Use -l with an anti-detection script to obfuscate Frida's presence, or try objection which includes built-in anti-detection.
Pitfall 3: Token expiry. Bearer tokens extracted during mitmproxy capture expire. Build token refresh logic into your scraper based on how the app authenticates (usually a login endpoint that returns a fresh token).
Conclusion
Mobile API extraction is a powerful technique precisely because it bypasses the browser-layer defenses that make web scraping expensive. Once you've mapped an app's API surface with Frida and mitmproxy, the actual data collection is often simpler and more reliable than browser automation.
Pair it with Evomi's mobile proxies for traffic that matches the expected source profile, and you have a complete mobile extraction stack. Test it free with Evomi's trial before committing to a volume plan.
Mobile apps often talk to the same backend as the web interface — but without the anti-bot layers guarding the web frontend. The app's API endpoints are usually cleaner, more structured, and less protected than what you'd scrape from HTML.
The barrier is SSL pinning: many apps verify that the certificate they receive matches a known-good certificate embedded in the app, preventing machine-in-the-middle inspection. Frida removes that barrier.
This guide covers SSL unpinning an Android app to map its API surface for scraping.
Prerequisites
Android device (rooted) or Android emulator
Python 3.11+
adb(Android Debug Bridge) installed and in PATHFrida server running on the device
mitmproxyfor traffic inspectionTarget APK installed on the device
Legal note: Only perform this on apps you have authorization to test, or on your own apps in a development environment. This guide is for security research and legitimate data collection from your own authorized targets.
Step 1: Set Up the Android Environment
Using an Emulator (Recommended for Getting Started)
Android Studio's emulator is the easiest path. Use a Google Play-free image (AOSP) for root access:
# List available system images sdkmanager --list | grep "system-images" # Create an emulator with a rootable image avdmanager create avd \ -n scraper_device \ -k "system-images;android-33;google_apis;x86_64" \ -d "pixel_6" # Start with writable system partition emulator -avd scraper_device -writable-system
Install Frida Server on the Device
# Find device architecture adb shell getprop ro.product.cpu.abi # → x86_64 # Download matching frida-server from github.com/frida/frida/releases # Example: frida-server-16.x.x-android-x86_64.xz xz -d frida-server-16.x.x-android-x86_64.xz adb push frida-server-16.x.x-android-x86_64 /data/local/tmp/frida-server adb shell chmod 755 /data/local/tmp/frida-server # Start frida-server (run as root) adb shell su -c "/data/local/tmp/frida-server &"
Verify it's working:
frida-ps -U # Should list running processes on the device
Step 2: Set Up mitmproxy
# Start mitmproxy mitmproxy --listen-port 8080 # Or mitmdump for headless/scripted inspection mitmdump -p 8080 -s log_requests.py
Install the mitmproxy CA certificate on the device:
# Route device traffic through your machine's mitmproxy adb shell settings put global http_proxy 192.168.x.x:8080 # Your machine's IP # Install mitmproxy cert # Navigate to http://mitm.it in the device browser # Download and install the Android certificate
At this point, HTTP apps work. HTTPS apps with SSL pinning will fail with a certificate error.
Step 3: SSL Unpinning with Frida
The Universal Unpinning Script
The most reliable starting point is the community-maintained universal unpinning script from Frida CodeShare. It hooks common SSL pinning implementations across most apps:
# Run the universal unpinning script against a target app frida -U \ -l https://codeshare.frida.re/@pcipolloni/universal-android-ssl-pinning-bypass-with-frida/ \ -f com.target.app
This hooks OkHttp3, TrustKit, Conscrypt, and the standard X509TrustManager implementation — covering the vast majority of Android apps.
A Manual Unpinning Script (When Universal Fails)
For apps with custom pinning implementations, you need to hook the specific pinning logic. First, identify it:
# Find pinning-related method names in the APK apktool d target-app.apk -o target_decoded grep -r "CertificatePinner\|TrustManager\|checkServerTrusted\|pinnedCerts" target_decoded
Then write a targeted hook:
// custom_unpin.js Java.perform(function() { console.log('[*] Starting SSL unpinning'); // Hook OkHttp3 CertificatePinner try { var CertificatePinner = Java.use('okhttp3.CertificatePinner'); CertificatePinner.check.overload('java.lang.String', 'java.util.List') .implementation = function(hostname, peerCertificates) { console.log('[+] OkHttp3 CertificatePinner.check bypassed for: ' + hostname); return; // Skip the check }; } catch(e) { console.log('[-] OkHttp3 not found: ' + e); } // Hook TrustManager try { var X509TrustManager = Java.use('javax.net.ssl.X509TrustManager'); var TrustManagerImpl = Java.use('com.android.org.conscrypt.TrustManagerImpl'); TrustManagerImpl.verifyChain.implementation = function( untrustedChain, trustAnchorChain, host, clientAuth, ocspData, tlsSctData ) { console.log('[+] TrustManagerImpl.verifyChain bypassed for: ' + host); return untrustedChain; }; } catch(e) { console.log('[-] TrustManagerImpl hook failed: ' + e); } console.log('[*] SSL unpinning complete'); });
Step 4: Capturing and Analyzing Traffic
Once unpinning is active, all HTTPS traffic from the app flows through mitmproxy in plaintext. Use mitmdump with a script to log and filter requests:
# log_requests.py — mitmproxy addon script import json from mitmproxy import http class APILogger: def __init__(self): self.api_calls = [] def request(self, flow: http.HTTPFlow) -> None: # Filter to API calls only if 'api.' in flow.request.host or '/v1/' in flow.request.path or '/v2/' in flow.request.path: call = { 'method': flow.request.method, 'url': flow.request.pretty_url, 'headers': dict(flow.request.headers), 'body': flow.request.text if flow.request.text else None, } self.api_calls.append(call) print(f"[API] {call['method']} {call['url']}") def response(self, flow: http.HTTPFlow) -> None: if 'api.' in flow.request.host: try: body = json.loads(flow.response.text) print(f"[RESPONSE] {json.dumps(body, indent=2)[:500]}") except: pass addons = [APILogger()]
Interact with the app manually, triggering the data you want to collect. You'll see every API call in the terminal.
Step 5: Replicating the API Calls
Once you've mapped the API surface, reproduce the calls in Python:
import httpx import json # Headers extracted from mitmproxy traffic HEADERS = { 'User-Agent': 'TargetApp/4.2.1 (Android 13; Pixel 6)', 'Authorization': 'Bearer YOUR_TOKEN_FROM_CAPTURE', 'X-App-Version': '4.2.1', 'X-Device-ID': 'extracted-device-id', 'Accept': 'application/json', 'Content-Type': 'application/json', } # Evomi mobile proxy for matching traffic profile PROXY = 'http://USERNAME:PASSWORD@mobile.evomi.com:2000' async def call_api(endpoint: str, params: dict) -> dict: async with httpx.AsyncClient( proxies={'http://': PROXY, 'https://': PROXY}, headers=HEADERS, timeout=30.0, ) as client: response = await client.get(endpoint, params=params) response.raise_for_status() return response.json() # Example: replicate a product search call async def search_products(query: str, location: str) -> list: data = await call_api( 'https://api.target-app.com/v2/products/search', {'q': query, 'location': location, 'limit': 50} ) return data.get('results', [])
Using Evomi's mobile proxies here matches the expected IP profile — real 4G/5G carrier IPs that mobile API backends treat as genuine app traffic, not datacenter or even residential IPs.
Common Pitfalls
Pitfall 1: Certificate not trusted at OS level. Android 7+ doesn't trust user-installed certificates for app traffic by default. The mitmproxy certificate needs to be installed as a system-level CA. On rooted devices:
# Get cert hash openssl x509 -inform PEM -subject_hash_old -in ~/.mitmproxy/mitmproxy-ca-cert.pem | head -1 # → e8d04e88 adb remount # Requires rooted device adb push ~/.mitmproxy/mitmproxy-ca-cert.pem /system/etc/security/cacerts/e8d04e88.0 adb shell chmod 644 /system/etc/security/cacerts/e8d04e88.0
Pitfall 2: App detects Frida. Some apps check for Frida's presence using process name scanning or port detection. Use -l with an anti-detection script to obfuscate Frida's presence, or try objection which includes built-in anti-detection.
Pitfall 3: Token expiry. Bearer tokens extracted during mitmproxy capture expire. Build token refresh logic into your scraper based on how the app authenticates (usually a login endpoint that returns a fresh token).
Conclusion
Mobile API extraction is a powerful technique precisely because it bypasses the browser-layer defenses that make web scraping expensive. Once you've mapped an app's API surface with Frida and mitmproxy, the actual data collection is often simpler and more reliable than browser automation.
Pair it with Evomi's mobile proxies for traffic that matches the expected source profile, and you have a complete mobile extraction stack. Test it free with Evomi's trial before committing to a volume plan.

Author
The Scraper
Engineer and Webscraping Specialist
About Author
The Scraper is a software engineer and web scraping specialist, focused on building production-grade data extraction systems. His work centers on large-scale crawling, anti-bot evasion, proxy infrastructure, and browser automation. He writes about real-world scraping failures, silent data corruption, and systems that operate at scale.



