diff options
Diffstat (limited to '2016/cellular_modems_33c3/33c3-modems.adoc')
| -rw-r--r-- | 2016/cellular_modems_33c3/33c3-modems.adoc | 528 |
1 files changed, 528 insertions, 0 deletions
diff --git a/2016/cellular_modems_33c3/33c3-modems.adoc b/2016/cellular_modems_33c3/33c3-modems.adoc new file mode 100644 index 0000000..e00627f --- /dev/null +++ b/2016/cellular_modems_33c3/33c3-modems.adoc @@ -0,0 +1,528 @@ + +Dissecting modern (3G/4G) cellular modems +========================================= +:author: Harald Welte, Holger Hans Peter Freyther +:copyright: Harald Welte, Holger Hans Peter Freyther (License: CC-BY-SA) +:backend: slidy +:max-width: 45em + +//include::33c3-modems.css[] + +== This talk + +* Our motivation +* A bit of History +* Selecting a device +* An unexpected surprise +* Firmware upgrade +* Outlook/Recommendations/Wishes + +== Motivation + +// 9 years of Osmocom? +// 3G and 4G development +// Hardware for decoding +* Implementing GSM specifications for the last decade (OpenMoko, Osmocom) +* 8 years since _Anatomy of Smartphone Hardware_ at 25C3 +* 7 years since OsmocomBB for GSM +* Used and built M2M devices using 2G modems at work +* so we're looking for a modem that can be used for +** our next-generation M2M/embedded devices +** testing/logging/tracing Osmocom 3G/4G network-side software +** building more tools to help understanding cellular technology + +== Cellular Modems in M2M + +image:images/sl6087_hw.png[height=300,role="gimmick_right"] + +* Assume you want to build a M2M device +* Classic approach to M2M/Embedded cellular: +** Cellular modem with AT commands over Serial/USB +** Main Processor runs M2M application +* if you run Application in Modem, you can save PCB space, power and BOM cost +** OpenAT by Sierra Wireless +*** Write C code using OpenAT APIs +*** Dynamically loaded into the RTOS +*** Runs without privilege separation, MMU +*** Protocol to multiplex AT, log, debug +*** Discontinued HW platform => Locked in +*** Various other limitations + +== Device requirements + +Our requirements for a good modem + +** Ability to run application code inside modem +** Avoid modem supplier vendor lock-in (EOL, ...) +** Get textual logging when handling messages +** Get a copy of the radio network messages and export to GSMTAP +*** Like Tobias Engels https://github.com/2b-as/xgoldmon[x-goldmon] +*** But for all GPRS, EGPRS, UMTS and LTE messages + +== Qualcomm DIAG protocol + +* Qualcomm DIAG in many products (DVB-H, GSM, ...) +* https://events.ccc.de/congress/2011/Fahrplan/attachments/2022_11-ccc-qcombbdbg.pdf[Presented] by Guillaume Delugre at 28C3 +* Simple HDLC frame (0x7e), cmd, data, CRC16 + +* Events, Logging, Command/Response +* Thousands of different message structures +* ModemManager, gsm-parser consume only a small fraction + +image:images/diag_frame.svg[width="90%"] + +== Selecting a device + +image:images/28c3_option_stick.png[width="30%",role="gimmick_right"] + +* Old Option Icon 225 stick exposes DIAG out of the box +* Quectel UC20 (2G+3G) expose DIAG by default +** but no LTE support +* Quectel EC20 (2G+3G+4G) expose DIAG by default +** 2G, 3G and 4G sounds quite nice +** EC20 not only a LGA solder module but also as mini-PCIe +*** convenient for early testing / prototyping without custom board + +image:images/ec20.png[height=300,role="gimmick_right"] + +* EC20 using a Qualcomm MDM9615 chipset +** Also used in the iPhone5 +** Almost no documentation on MDM9615 available +** Still, a good candidate for starting our research... + +// Erst ein mal EC20 und sagen wieso es interessant ist +// und dann, dass es Linux hat.. um dann ein Block diagram +// zu haben? + +[role="change_topic"] +== An unexpected surprise + +== Firmware update, hints of Linux + +* Got a firmware upgrade to fix stability / bugs +* Looks like it contains traces of Linux? +* Looks like it uses fastboot for the update +* Other people have already found Linux in MDM9615 based products (e.g https://media.defcon.org/DEF%20CON%2023/DEF%20CON%2023%20presentations/DEFCON-23-Mickey-Shkatov-Jesse-Michael-Scared-poopless-LTE-and-your-laptop-UPDATED.pdf[Mickey Shkatov] at DEFCON 23) +* But why would there be Linux inside a Modem? +** Qualcomm is known for their REX/AMSS on Hexagon baseband ?!? +* And if it contains Linux, GPL requires them to mention that, include + License text and provide source code ?!? + +== GPL compliance + +* No written offer, let's see if it runs Linux +* Armijn Hemels `gpltool.git` has `unyaffs` to unpack yaffs +* `strings`, etc. clearly reveal Linux, glibc, busybox +** other interesting strings like `AT+QLINUXCMD=?` show up +* The fun and exploration begins... +** technical analysis (serial console, firmware reversing, ...) +** legal enforcement to get source code of GPL/LGPL components (Harald is founder of http://gpl-violations.org[gpl-violations.org]) + +== Hardware based analysis + +* mPCIe modules often expose additional signals like PCM audio on + non-standard pins +* existing PC/embedded mainboards don't use those signals +* create Osmocom mPCIe-breakout board to access those signals +* https://osmocom.org/projects/mpcie-breakout/wiki + +image:images/mpcie_breakout.jpg[width="70%"] + +== Serial Console + +* EC20 solder module documents DBG_UART pinout, but not all modules + have it enabled? +* serial console is at 1.8V, but the 1.8V supply is not accessible (so + not easy to add external level shifter / Vref) +* create Osmocom multi-voltage USB-UART with selectable 1.8, + 2.3, 2.5, 2.8, 3.0 and 3.3V logic level + +image:images/mv_uart.jpg[width="40%",role="gimmick_right"] + +* https://osmocom.org/projects/mv-uart/wiki +* root password (DES hash): `oelinux123` + +== Retro-fitting Serial Console to mPCIe module + +* unfortunately the DBG_UART on the LGA module solder pads is not + exposed to mPCIE +* some soldering required to retro-fit a 2.54mm header: + +image:images/ec20_uart.jpg[width="70%"] + +== GPL compliance + +* Linux basis created by Qualcomm and used by Quectel +** https://wiki.codeaurora.org/xwiki/bin/QLBEP/ +** Many branches, releases, which to use? + +[quote, Tonino Perazzi] +I tried instruction above to build yaffs2 for MDM9615, so I downloaded source `M9615AAAARNLZA1611161.xml` but during compilation I faced some libs that are missing such as libQMI and acdb-loader.. + +image:images/qualcom_many_releases.png[width="80%"] + +== GPL compliance + +[qanda] +Asking for the complete and corresponding source:: +[quote,Quectel] +** The source code of Qflash tool in Linux is attached, [...] +[qanda] +Asking again for the complete and corresponding source:: +[quote,Quectel] +We never been in legal dispute and we always make sure to understand IPR ahead of using technology belonging to third party. + +image:images/quectel_ipr.jpg[width="100%"] + +== GPL compliance + +[qanda] +Asking for the complete and corresponding source:: +[quote,Quectel] + We appreciate the efforts that your client had put into the open source +project netfilter/_iptable_. However, [...] *your client does not have the right to +empower the copyright*. We think software netfilter/iptable is built on +the code operating system _GUN_/Linux, thus subject to GPL terms, where FSF +requires that each author of code incorporated in FSF projects either +provide copyright assignment to FSF or disclaim copyright. Therefore, +It seems that *your client does not have the copyright on netfilter/iptable.* + + + +As one of the leading providers of wireless solution, *Quectel is always +respectful IPR*. We would like to compliant with GPL and do some necessary +statements，including a disclaimer or appropriate notices. Under the terms +of GPL, we would like to dedicate Kernel code of EC25x to free software +community. + +== GPL compliance + +[qanda] +Asking for the complete and corresponding source:: +[quote,Quectel] + Many thanks for your detailed explanations GPL/LGPL license terms and the practical methods. I will carefully study your suggestions again and find a proper way to open GLP/LGPL licensed software. Basically, we will simply provide a tarball of open source for download at this time. And release the git repositories in next step. + +[qanda] +Asking for the complete and corresponding source:: +[quote,Quectel] + We are always willing to achieve GPL compliance. + +[qanda] +Asking for the complete and corresponding source:: +[quote,Quectel] + So we need some time to know of all things and construct the Open Source projects. Within a short time, we cannot construct a perfect web site to present Open Source things now. However, we will continue to do like that. + +== GPL compliance + +[qanda] +Your tarball is missing some files:: +[quote,Quectel] +We have issued all GPL licensed source code. +*We have no the xt_dscp file in the project, and nor Qulacomm*. It must be +caused by your compilation environment. +If you have more question or problem during the development with Quectel +module, please add my Skype ID (XXXXX), I will continue to support you +on Skype. + +*The email will not discuss the compiling issue any more.* + + + +== GPL compliance + +* ... many months later +** we have received various source tarballs +** they contain not only GPL/LGPL code but other FOSS code (thanks!) +** full license compliance still not achieved, but improving... +* Sierra Wireless Legato is a positive example of a competitor +** they not only provide the OE/Linux source but extensive +documentation! +** but they try to lure customers into a proprietary Legato framework, +and thus again vendor-lock-in :( + +image:images/legato_flash.png[width="80%"] + +[role="change_topic"] +== MDM 9615 HW and SW + + +== Qualcomm Hardware + +* Qualcomm MDM9615 chipset +* Used in the iPhone 5 and automotive +* Modems like Quectel EC20, Sierra Wireless MC7355 +* No public HW documentation?! +* Either not many people study it or are not allowed to share? + +== MDM 9615 HW Overview + +* ???? +// Block diagram? +// Listing of interfaces. +// Show it is a highly complex SoC... with even more things +// that are unknown.. device tree file, peripheral, etc + + +== How to access the system? + +* serial console requires soldering re-work and is slow +* easy mechanism to get shell and transfer files from/to target +* Android `adbd` present on the modem but not exposed via USB +* it's possible to re-configure the Linux kernel Android USB Gadget: +** `AT+QLINUXCMD="/usr/bin/usb_uartdiag"` +** device re-enumerates with different composite USB interfaces +* Linux kernel driver on host needs patching (static interface + mapping assumption) +** patches available in `quectel-experiments.git`, documented in wiki + + +== MDM 9615 AP SW Overview + +image:images/gandroid_logo.png[height=200,role="gimmick_right"] + +The software stack seems to be called *Qualcomm LE* + +* Android Bootloader +* Android Linux kernel +* Android Debug Bridge (adb) +* but: GNU libc, busybox userland +* Using OpenEmbedded to build images +* Developed and maintained by Qualcomm + + +== Qualcomm Linux kernel overview + +* Qualcomm Android Linux kernel +* Huge changes compared to mainline `git diff -w | wc -l` +** `v3.0.21` in EC20: 1.5 million lines +** `v3.18.20` in EC25: 1.9 million lines +* Expected: CPU + peripheral drivers +* Less expected: +** smem_log (shared memory logging) +** ipc_log (inter-processOR communication) +** remote spinlocks + +== Qualcomm Linux kernel subsystems + +Some of the Qualcomm-specific kernel sub-systems + +[cols="20%,80%"] +|=== +|SMD|Shared Memory Device +|IPC|Inter Processor Communications +|RMNET|Remote Network +|BAM|Bus Access Manager +|IPA|Internet Packet Accelerator +|DIAGFWD|DIAG Forwarding +|AF_MSM_IPC|Socket family for Qualcomm IPC +|=== + +== Qualcomm LE System Architecture + +image:images/qualcomm_le.svg[width="50%",role="gimmick_right"] + +* simplified block diagram +* USB interface fully controlled by Linux AP +** very complex Qualcomm Android USB Gadget +** some endpoints mapped to SMD queues +** other endpoints handled by _regular_ Linux +** GPS NMEA takes completely different path than AT commands, despite +both being serial ports? +** DIAG and QMI handled in more complex ways + +== DIAG in Qualcomm LE + +* DIAG interface of Modem exposed on SMD +* diagfwd distributes messages between USB, SMD and `/dev/diagchar` +* Linux userspace processes don't use syslog, but diag msg for logging via `libdiag.so` + +image:images/diag.svg[width="100%"] + +== QMI in Qualcomm LE + +every `rmnet` data device has associated QMI control + +* on your Linux PC: `qmi_wwan` and `/dev/cdc-wdm` +* on Qualcomm LE modem: `/dev/smdcntlN`, multiplexed by `qmuxd` + +image:images/qmi_smd_qmuxd.svg[width="100%"] + +== Tools for analysis + +We created some tools to help our analysis + +* used OE to build matching `opkg` and OE packages for `socat`, `lsof`, `strace` +* FOSS programs for the Linux AP linked against proprietary `libqmi-framework.so` +** `qmi_test`: Simple program to read IMEI via QMI +** `atcop_test`: Test program to implement AT commands in Linux userspace +* 100% FOSS programs +** `qmuxd_wrapper`: LD_PRELOAD wrapper for tracing between `qmuxd` and QMI clients +** `libqmi-glib` transport support for `qmuxd` (work in progress) +** `osmo-qcdiag`: Host tool for obtaining DIAG based logs from Linux programs + QMI traces, decoded via `libmi-glib` + +== Userspace programs + +We found a bunch of proprietary Linux userspace programs + +[cols="20%,80%"] +|=== +|`adbd`|Implements Android Debug Bridge +|`atfwd_daemon`|Implement Quectel-Specific AT Commands +|`quectel_daemon`|?; various ASoC related bits +|`qti`|? +|`mbim`|Mobile Broadband IF Model (translates MBIM to QMI) +|`QCMAP_ConnectionManager`|runs linux-base WiFi AP/router with LTE backhaul +|`quec_bridge`|reads GPS NMEA from `/dev/nmea` and writes it to `/dev/ttyGS0` +|=== + +[role="change_topic"] +== Funny bits + pieces + +== Funny AT commands + +* `AT+QLINUXCMD`, e.g. switch usb config to get adb +** arbitrary shell commands executed as root on r/w rootfs! +* `AT+QFASTBOOT`, switch to the bootloader +* `AT+QPRINT`, print dmesg +* AT for `system("echo mem > /sys/power/state")` + +== How many processes does it take to reboot a system? + +* `rebootdiagapp` registers DIAG command (cmd code 0x29) +** spawns thread that runs `system("qmi_simple_ril_test input=/tmp/reset")` +** `system("echo 'modem reset' > /tmp/reset")` +*** makes `qmi_simple_ril_test` send a QMI message to modem +** `system("rm /tmp/reset")` +** writes "REBOOT" to `/dev/rebooterdev` this time using `fwrite()`! +* `reboot_daemon` reads `/dev/rebooterdev` + +---- +read_count = read(pipe_fd,buf,MAX_BUF-1); +/* if read REBOOT_STR, then call reboot */ +if(strncmp(buf,REBOOT_STR,strlen(REBOOT_STR)) == 0) { + debug_printf("going for reboot\n"); + printf("reboot-daemon: initiating reboot\n"); + system("reboot"); +} +---- + +== C programs that look like shell scripts + +* strings /usr/bin/quectel_daemon + +---- +echo "nau8814-aif1" > /sys/devices/platform/soc-audio.0/tx_dai_name +cp -f /cache/usb/qcfg_usbcfg /etc/; cp -f /cache/usb/usb /etc/init.d/ +echo 90 >/sys/kernel/debug/pm8xxx-pwm-dbg/0/duty-cycle +pkill -f "/bin/sh /usr/bin/nmea_demon.sh" +ps ef | grep "quec_bridge /dev/nmea /dev/ttyGS0" | grep -v grep +cd /cache/ufs;ls +---- + +[role="change_topic"] +== Firmware upgrade + +== recovery and applypatch + +* Qualcomm uses https://android.googlesource.com/platform/bootable/recovery.git/+/android-4.0.4_r2.1[recovery.git] from Android ~4.0 +* Updates are zip files with deltas, SHA1+RSA +* recovery started on boot, drives applypatch +---- +// Look for an RSA signature embedded in the .ZIP file comment given +// the path to the zip. Verify it matches one of the given public +// keys. +---- + +== Qualcomm EC20 firmware upgrade + +image:images/redbend.png[width="30%",role="gimmick_right"] + +* Based on the recovery.git code +* But for some reason using RedBend for the update (legacy?) +* RSA still linked into the binary but not used +* RedBend used by many more companies and systems (e.g. Quectel UC20, automotive) + + +== RedBend (delta update) software + +* Used in OMA DeviceManagement as well? (e.g. https://www.blackhat.com/docs/us-14/materials/us-14-Solnik-Cellular-Exploitation-On-A-Global-Scale-The-Rise-And-Fall-Of-The-Control-Protocol.pdf[Mathew Solnik]) +* Lots of starring at hexdumps, lots of help from Dieter Spaar +* Created tools to partially extract and create .diff files +* Heavy in pointers/offsets, not robust +* Crashes on crafted files +* Not cryptographically signed! + +image:images/delta_header.png[width="80%"] + + +== Firmware upgrade overview + +image:images/upgrade_process.svg[width="55%",role="gimmick_right"] +//[source] +---- +$ strings atfwd_daemon | egrep "wget|QCMAP|fota|update.z" + +... QCMAP_ConnectionManager /etc/mobileap_cfg.xml n n fotanet +/usr/bin/wget -T 20 -t 3 %s -O %s +mv %s %s && mkdir -p /cache/fota && echo %s > %s +/cache/fota/ipth_config_dfs.txt +rm -rf /cache/fota /cache/recovery /cache/update.zip +Start download fota for update.zip +---- + +* atfwd_daemon can be asked to start upgrade +* Configure APN, specify URL, store result to update.zip +* Add status and reboot to recovery +* Apply update.zip and reboot + +== Recommendation to modem vendors + +* It is great to have an open and accessible Qualcomm based modem for + further research and developing custom applications/extensions +* Security issues (particularly unverified FOTA) must be fixed +* We need security from attackers _without locking out the user/owner_ +** If vendors introduce verified boot and/or FOTA, allow owner specified keys! +* Please keep it open, good for learning and many applications +* Allow owners to modify the software of their device +* Secure the FOTA upgrading with owner specified keys + +== Status and Outlook + +* Status today +** Osmocom wiki with all our findings public now! +** debug tools (`osmo-qcdiag`, LD_PRELOAD wrapper, `qmi_test`, etc.) released +** mpcie-breakout + mv-uart released + available +** `libqmi-glib` integration WIP +* Outlook +** we hope to grow documentation in wiki +** please help us out: read code, play with devices + update wiki +** OE/opkg package feed planned +** aim is to have 100% FOSS userland on Cortex-A5 + +== Unrelated Announcement + +* Osmocom project has gained support for 3G/3.5G during 2016 +* Osmocom suffers from lack of contributions :( +* We want to motivate more contributions +** _Accelerate 3.5G_ programme provides 50 free 3.5 femtocells to contributors +** tell us how you would use your free femtocell to improve Osmocom +** Call for Proposals runs until January 31st, 2017. +** see http://sysmocom.de/downloads/accelerate_3g5_cfp.pdf + +== Questions + +* Questions? + + + +== Links + +* Our results / hacks +** https://osmocom.org/projects/quectel-modems +** git://git.osmocom.org/quectel-experiments.git +** git://git.osmocom.org/osmo-qcdiag.git +** ftp://ftp.osmocom.org/quectel (mirrored) +* Collection of links for further study +** ftp://ftp2.quectel.com/OpenSrc/ +** https://wiki.codeaurora.org/xwiki/bin/QLBEP/ +** https://events.ccc.de/congress/2011/Fahrplan/attachments/2022_11-ccc-qcombbdbg.pdf +** https://media.defcon.org/DEF%20CON%2023/DEF%20CON%2023%20presentations/DEFCON-23-Mickey-Shkatov-Jesse-Michael-Scared-poopless-LTE-and-your-laptop-UPDATED.pdf +** https://github.com/2b-as/xgoldmon +** https://www.blackhat.com/docs/us-14/materials/us-14-Solnik-Cellular-Exploitation-On-A-Global-Scale-The-Rise-And-Fall-Of-The-Control-Protocol.pdf |