diff options
| -rw-r--r-- | 2021/osmodevcall-pysim_shell/pysim_shell.adoc | 242 |
1 files changed, 242 insertions, 0 deletions
diff --git a/2021/osmodevcall-pysim_shell/pysim_shell.adoc b/2021/osmodevcall-pysim_shell/pysim_shell.adoc new file mode 100644 index 0000000..d6c9286 --- /dev/null +++ b/2021/osmodevcall-pysim_shell/pysim_shell.adoc @@ -0,0 +1,242 @@ +pySim-shell - next genreation SIM configuration tool +==================================================== +:author: Harald Welte <laforge@gnumonks.org> +:copyright: 2021 by Harald Welte (License: CC-BY-SA) +:backend: slidy +:max-width: 45em + +== why program your own SIM cards? + +* because you want to operate your own cellular network (2G, 3G, 4G, 5G) +** whether FOSS (Osmocom CNI, srsLTE, nextepc/open5gs, free5gc, OpenBTS, yateBTS, ...) +** but also proprietary (e.g. Amarisoft) +* 2G can use random 3rd party cards, but it's not really convenient in most cases +* Anything >= 3G requires you to know the card-specific secrete keys +** in reality that translates to having your own programmable SIM cards + +== some smartcard terminology + +* card filesystem +** *MF* (Master File): The root directory +** *DF* (Dedicated File): A subdirectory +** *ADF* (Application Dedicated File): Directory of an application (like USIM, ISIM) +** *EF* (Entry File): A regulare file +*** *Transparent EF*: An unstructured file +*** *Linear Fixed EF*: An file consisting of fixed-length records + +== specs vs. proprietary + +* SIM cards are fully specified by a combination of ISU, ETSI and 3GPP specs +* this covers only the operation after the card has been issued, e.g. +** reading and writing files accessible without PIN auth, after PIN auth +** performing GSM authentication or UMTS AKA +* it does _not_ cover how the card is issued/provisioned +** secret key material (Ki / K / OP / OPc) is not readable from the card +** it is an implementation detail on how the card manufacturer writes those to the card +* this leads to the need for card-specific support / code in software like pySim + +== pySim "classic" + +* originally developed in 2009 by Sylvain Munaut +* Linux/FOSS tool for configuration of SuperSIM/MagicSIM +* replicated what proprietary vendor tools did to configure IMSI, Ki, ... +* those ultra-cheap Chinese SIM cards at the time (from the SIM card cloning days of COMP128v1) shipped with binary-only windows programs + +== pySim "classic" fundamentals + +* library code +** _transport_ - how to exchange APDUs with card +** _commands_ - encoding of ISO7816-4 / TS 51.011 / ETS 102 221 commands +** _cards_ - classes implementing various different programmable SIM cards +* executable programs +** `pySim-prog` for card programming +** `pySim-read` for card reading + +== pySim transports + +* transport - how to exchange APDUs with card +** _serial_ transport for serial readers (UART Rx+Tx connected to I/O) +** _pcsc_ transport for pcsc-lite supported readers (e.g. USB CCID) +* some more exotic options added later on by Vadim Yanitskiy: +** _calypso_ for using L1CTL to talk to OsmocomBB on Ti Calypso phone +** _modem_atcmd_ for using AT+CSIM commands talking to a cellular modem + +== advanced pySim "classic" features + +* batch programming of cards, using CSV file input +* even with ability to execute external tool for driving motorized card readers + +== Limitations of pySim "classic" + +* supports only very few files/parameters that can be programmed +** IMSI, MSISDN, Ki and SMSP was sufficient for making classic GSM SIMs work on your own network +** today, people want to configure VoLTE/IMS parameters, 5G SUCI stuff, etc. +* adding more fields to pySim-classic makes it unusable: Who wants to specify 250 different fields as command line arguments? + +== Usability of pySim "classic" + +* it was designed to program a batch of cards with defaults and random keys +* there are all kinds of 'automagic' bits happening if not all fields are specified +* pySim at least originally assumed that you program a full card, and not that you perform incremental small changes +* no user manual, very limited RADME and wiki instructions + +== the road to pySim-shell + +What to do about the user interface? + +* GUIs are bad, they are not really scriptable/automatizable +* GUIs are hard to portable, test automatically, etc. +* people setting up FOSS cellular programs most likely use bash/zsh/tcsh on daily basis, so they are familiar with the concept of interactive shells / CLIs +* so let's do a CLI / shell / REPL! + +== pySim-shell and `cmd2` + +* uses `cmd2` python module for the shell +** extremely powerful framework for building CLI/shell type interfaces +** allows us to focus on implementing logic (like individual commands), not the shell itself +* supports output redirection to local files +* supports piping output through external programs ("| grep foo") +* supports `argparse` to define commands and their options +* allows to dynamically register and unregister commands +* allows full control over tab completion items. + +== pySim-shell UI fundamentals + +* some global commands for things like PIN/CHV verification, enabling, disabling, unblcoking +* you `select` your way through the card filesystem, like 'cd' on bash/zsh/... +** tab-completion for [known] file names + +.Selecting a file +---- +pySIM-shell (MF/ADF.USIM)> select EF.IMSI +{ + "file_descriptor": { + "shareable": true, + "file_type": "working_ef", + "structure": "transparent" + }, + "file_identifier": "6F07", + "proprietary_info": { + "proprietary_D0": "20", + "proprietary_D2": "0F" + }, + "life_cycle_status_int": "operational_activated", + "security_attrib_ref_expanded": "6F0603", + "file_size": 9, + "short_file_id": "38" +} +pySIM-shell (MF/ADF.USIM/EF.IMSI)> +---- + +== pySim-shell reading/writing + +* context-sensitive 'action' commands depend on the type of file selected +** `read_binary`, `update_binary`, ... for transparent EF +** `read_record`, `update_record`, ... for linear fixed EF + +.Reading and updating a transparent EF +---- +pySIM-shell (MF/ADF.USIM/EF.IMSI)> read_binary +089910070000400300 +pySIM-shell (MF/ADF.USIM/EF.IMSI)> update_binary 089910070000400301 +pySIM-shell (MF/ADF.USIM/EF.IMSI)> read_binary +089910070000400301 +---- + +== pySim-shell decoding/encoding + +* working with hex-encoded binary data is not very convenient +* each specified file in the filesystem is represented by a python class +** class can provide encoder + decoder methods to go from binary to JSON and vice-versa +* some shell commands combine reading a file from the card + decoding / encoding +** `read_binary_decoded`, `write_binary_decoded`, ... for transparent EF +** `read_record_decoded`, `write_record_decoded`, ... for linear fixed EF + +.Reading and updating a decoded transparent EF +---- +pySIM-shell (MF/ADF.USIM/EF.IMSI)> read_binary_decoded +{ + "imsi": "901700000043000" +} +pySIM-shell (MF/ADF.USIM/EF.IMSI)> update_binary_decoded '{"imsi": "901700000043999"}' +pySIM-shell (MF/ADF.USIM/EF.IMSI)> read_binary_decoded +{ + "imsi": "901700000043999" +} +---- + +== pySim-shell editing of single parameters + +* many files contain a lot of parameters +* entering all of them as a single line in `update_binary_decoded` becomes rather cumbersome +* `edit_binary_decoded` and `edit_record_decoded` to the rescue +** reads data from the card +** decodes it to JSON +** spawns whatever is your favorite text editor +** you do your editing, save + exit +** re-encodes the modified JSON +** writes the result to the card + +== pySim-shell `export` + +* one-shot command to iterate over all [known] files below the current DF and dump them +* output format is structured in a way that you can use it as a pySim-shell script +* results in easy method to backup and restore a card + +NOTE:: only the ETSI/3GPP standard files that are covered by pySim classes are exported. There may be any +number of proprietary additional files on the card. Particularly K/OP/OPc or OTA keys are not exported this +way + +== pySim-shell advanced cmd2 usage + +* `run_script`: execute any text file as script inside the pySim-shell +* `run_pyscript`: execute a python script with access to all of the state / classes / methods +* `ipy`: run an interactive IPython session +* `py`: run an interactive python interpreter +** `card` represents the `pySim.cards.Card` instance +** `rs` represents the `pySim.filesystem.RuntimeState` + +.Using interactive python shell with access to pySim-shell state +---- +pySIM-shell (MF)> py +Python 3.9.2 (default, Feb 28 2021, 17:03:44) +[GCC 10.2.1 20210110] on linux +Type "help", "copyright", "credits" or "license" for more information. + +End with `Ctrl-D` (Unix) / `Ctrl-Z` (Windows), `quit()`, `exit()`. +Non-Python commands can be issued with: app("your command") + +>>> rs.selected_file +<pySim.filesystem.CardMF object at 0x7f97952f3070> +>>> rs.selected_file.desc +'Master File (directory root)' +---- + + +== Other recent pySim developments + +* introduction of `sphinx` for generating documentation +* adding API documentation at least for all of the *new* classes/methods that are used to build pySim-shell, + and which you need to extend to cover more files +* automatic generation of pySim-shell command reference from `argparse` instances + +== Call for Contributions + +* pySim-shell is already very usable, but needs your help! +** most files still need encoder + decoder methods to be implemented +** we need automatic testing of encoders + decoders +** we want JSON schema for the JSON representation of each file +** we want to have batch programming in pySim-shell + +So if you know a bit of python and care about SIM cards, please send your patches! + +== Further Reading + +* https://osmocom.org/projects/pysim/wiki[pySim project page on osmocom.org] +* https://git.osmocom.org/pysim/about/[pySim source code / git repository] +* https://media.ccc.de/v/36c3-10737-sim_card_technology_from_a-z[Video of talk "SIM card technology from A-Z"] + + +== EOF + +End of File |