Information Security Stack Exchange is a question and answer site for information security professionals. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a bunch of SMSs in Chinese from a forensic extraction of a blackberry bold 9780 via Cellebrite UFED. While messages in English are clear-text readable (making me feel that this has nothing to do with encryption), I simply cannot represent these (Chinese) messages. It's obviously a problem with UFED's analysis software in the long term, but in the short term, I just want to know how I can make these messages readable. These Chinese texts are readable on the phone itself, but I don't have that anymore.

Here's three texts (via an export to html):

N mwyûR¨geu5cÐN:`¨g R¡ÿ�0�0�9�0�5�3�2�2�1�3�0�0�8�9S÷xW(�2�0�1�3^t�0�3g�0�6eå�2�3�:�3�8bèbSÇ`¨vbKg:ÿ÷SÊeöVÞu50


�(�1�/�3�)NûOUbRONýDg,g`gFÿN mwN¤Y'mwY[fbê�0�6^t_�R0 yÁRß¡gCNN ^D`;Ásí0NågeÇ|û~ßþz¾nÿNÎUFN0N¤Y'mwY0

Can anyone decode the three messages above? I will give you a million props if you can tell me how.

share|improve this question
This seem UTF8 or UTF16 viewed in Ascii console. There miss some values replaced there by an interrogation point: . Decoding this not possible in this state. we need original raw dumper in hex maybe. – F. Hauri Apr 7 '13 at 10:51
Agreed. We need the raw hex dump in order to get into this. – Polynomial Apr 7 '13 at 10:57
Could you export to raw or hexadecimal ?? – F. Hauri Apr 7 '13 at 11:01
For other texts I'll probably have to copy/paste the hex and encode it by hand. Does anyone recommend a tool for doing so (once we figure out the encoding)? – Dan Apr 8 '13 at 8:21

The source data is clearly UTF-16. In your post it is displayed interpreted as ISO-8859-1, and has become slightly mangled in the process - some bytes have been converted to .

There are also either some missing bytes caused by the copy-paste, or there are some control bytes in the middle of the sequence, which disrupt the two-bytes-per-character flow. Hopefully the former - can you post the raw bytes (eg as hex dump)? If this is the case, you should be able to just decode them as UTF-16 (probably UTF-16LE).

For what it's worth, trying to recover the original bytes, with interpreted as null (as the most common byte in UTF-16) using Python:

>>> data= ur'''N mwyûR¨geu5cÐN:`¨g R¡ÿ�0�0�9�0�5�3�2�2�1�3�0�0�8�9S÷xW(�2�0�1�3^t�0�3g�0�6eå�2�3�:�3�8bèbSÇ`¨vbKg:ÿ÷SÊeöVÞu50\0\0Y)s+UFWÎmilýQÀl4VheàPOS[cÿQí¤Áx�8�0�1�1bèbS�4�0�0�8�2�0�2�6�6�7bSÖN÷P<�2�5�8�0QCQÀl4VhN�Sð00milýs¯OÝ0\0\0�(�1�/�3�)NûOUbRONýDg,g`gFÿN mwN¤Y'mwY[fbê�0�6^t_�R0 yÁRß¡gCNN ^D`;Ásí0NågeÇ|û~ßþz¾nÿNÎUFN0N¤Y'mwY0'''
>>> bytes= data.replace(u'\uFFFD', u'\0').encode('iso-8859-1')
>>> print bytes.decode('utf-16le', 'replace')
>>> print bytes.decode('utf-16be', 'replace')

Then pick the characters you find most fitting from each line of:

⁎睭ﭹꡒ敧㕵큣㩎ꡠ⁧ꅒÿ0090532213008匹磷⡗㈀ ㄀㌀瑞 ㌀g0收å23:3戸拨읓ꡠ扶杋Z号旊囶痞〵ぜぜ⥙⭳䙕칗業ﵬ쁑㑬桖佐孓」솤x801戱拨S400820266户홓㱐㈀㔀㠀 䍑쁑㑬桖N〰業ﵬ꽳�尰尰0(1/3丩俻払佒﵎杄本杠f⁎睭ꑎ❙睭孙扦ê0帶彴刀‰셹�枡乃⁎䑞㭠珁ロ敧糇绻ﻟ빺ョ칎䙕ぎꑎ❙睭す
丠海移动来电提为您朠务＀  㤀 㔀㌀㈀㈀㄀㌀  㠀㥓在2013年03最 㙥㈀㌀㨀㌀㡢叇您癢䭧㫿쩥�㔰尰尰天猫商城浩泽净水器无偏卛揿凭꓁砀㠀 ㄀ㅢ匀㐀  㠀㈀ ㈀㘀㘀㝢取价值2580元净水器一台〰浩泽环保ぜぜ ⠀㄀⼀㌀⥎ﭏ啢剏份䑧Ⱨ恧䛿丠海交大海奛晢 㙞瑟R〠私募ꅧ䍎丠幄总셳以来일ﭾ�窾滿从商丰交大海夰

eg something like:


which is as near as I can get to intelligible given the mangled input.


Here's a hex dump of a longer SMS:

OK this is straightforward, it's UTF-16BE (with a couple of lead control bytes and some trailing ones):

>>> print '0848674376CA90FD4E0D80FD4FDD969C76845BA26237300262404EE573B05728770B6765FF0C516C53F85B81613F9AD86210672C9AD84E8E6052592971364EF7683C53BB57F9517B6276630151764ED676844F9B5E945546FF0C4ECE957F671F770BFF0C7EE77EED51CF5C11548C605259297136768454084F5C657091CF548C53606BD43002000000084900'.decode('hex').decode('utf-16be')

The corresponding encoded/analyzed SMS.Body was:

OK, so that is another string of UTF-16BE bytes that has been interpreted as ISO-8859-1. Recovering the control characters from your post we get:

>>> u'q\x8ae\x87cw:N\nk!u5\x8b\xddN_T\x8cO`\x8b\xf4N\x86\xff\x0c`RY)q6v\x84\x88LN:\x8b\xa9QlS\xf8\x8b\xa4N:\xff\x0c`RY)q6N\nf/b\x11N\xecv\x84b\x18ueT\x08O\\O\x9b^\x94UF\xff\x0cZ\x03T\xc8T\xc8[\xf9`RY)q6ge\x8b\xf4\xff\x0cN\xc5N\xc5f/NN*fn\x90\x1a[\xa2b7\xff\x0c\x80\x0cN\x14f/kc^8gCv\xca\x90\xfdN\n\x80\xfdO\xdd\x96\x9cv\x84[\xa2b70\x02b@N\xe5s\xb0W(w\x0bge\xff\x0cQlS\xf8[\x81a?\x9a\xd8b\x10g,\x9a\xd8N\x8e`RY)q6N\xf7h<S\xbbW\xf9Q{bvc\x01QvN\xd6v\x84O\x9b^\x94UF\xff\x0cN\xce\x95\x7fg\x1fw\x0b\xff\x0c~\xe7~\xedQ\xcf\\\x11T\x8c`RY)q6v\x84T\x08O\\ep\x91\xcfT\x8cS`k\xd40\x02\n'

Undoing the misdecoding:

>>> print _.encode('iso-8859-1').decode('utf-16be', 'ignore')

As you can see, the second part of this string is the complete message of which the above was a partial hex-encode.

The first part (before the 个普通客户...) is a mystery. It decodes as valid UTF-16LE (ie no need to 'ignore'), which would normally be an indication that the byte alignment had slipped (again, maybe as a result of character mangling from posting it to SO?). The result of this decoding even fits mostly inside the han character block, but it is:


which from my limited understanding of Chinese text appears to be gibberish.

I'm not too impressed with this Cellebrite thing, especially if it is including those control codes directly in XML! (This is not well-formed.)

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.