I'm playing a bit with assembler and I decided to test out how metasploit encodes a payload into Unicode uppercase characters as assembly instructions which are meant to be decoded by itself. So I have written simple shell code consisting of 10 nops and a ret. I placed it into a file 1.payload.
I'm using metasploit:

Framework: 4.17.4-dev-4744fb01d9c4234a60c35dc78b9a8710fc19fa01
Console : 4.17.4-dev-4744fb01d9c4234a60c35dc78b9a8710fc19fa01

on Windows 10.

I tried to encode my payload like this (from cmd.exe):

msfvenom -a x86 --platform windows -p generic/custom 
PAYLOADFILE=c:\test\1.payload -e x86/unicode_upper -o c:/out/encoded.bin

And I get this error:

Found 1 compatible encoders
Attempting to encode payload with 1 iterations of x86/unicode_upper x86/unicode_upper failed with An encoding exception occurred.
Error: An encoding exception occurred.

I do not have any zero bytes in my "payload" it's just 10 nop instructions, and a ret instruction. The encoder for alphs works fine but for some reason it generates weird non-Latin characters at the beginning of the output. Can someone help, do I use it right? How can I encode my x86 assembler payload in Unicode uppercase instructions?

1 Answer 1


A workaround for it is to leave off the encoder selector -e x86/unicode_upper. Thus, the msfvenom will try some encoders up they find some that work with the corresponding bad characters.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .