>> Hi, welcome to Cybrary. My name is Sean Pierce.
I'm the subject matter expert for
introduction to malware analysis.
Today, we're going to be covering
calling conventions and X86 assembly.
What exactly are calling conventions?
In simple terms,
when a function call is made with parameters,
the assembly code will do a series of
pushes to push those parameters
onto the stack in reverse order.
Then we'll execute the call instruction
and the call instruction as its operand.
It's second part of the instruction is
>> the location of the function call.
>> In most code,
the caller is responsible for cleaning up the stack.
The instruction right after
that call instruction is usually
a sub ESP instruction
where the stack is being cleaned up.
Because the caller had
just pushed a bunch of values onto the stack.
If it pushed three values
on the stack and pop three values,
that would work just as well.
I'm sorry, it's not sub ESP.
It's like at ESP.
The other major calling convention, is standard call.
It's actually something made up by Microsoft.
It's how it's API works.
It works pretty much the same way,
where it pushes the parameters in reverse order,
and the only differences is,
the callee is responsible for cleaning up the stack.
You'll see in a standard call
there is, push, push, push,
and then a call and then right under the call is
no other parameters and/or no other
>> stack instructions like
>> add ESP or add something to ESP,
or pop, pop, pop or whatever else.
Inside that function call,
that function cleans up the stack.
It does pop, pop, pop at the end or it'll do a RET
for or whatever where it is explicitly saying,
''I am going to fix the stack.''
You might think, that doesn't matter very much.
You're right, it doesn't,
the compiler just need to choose one,
be consistent or not.
For instance, it can do some pretty cool optimizations,
like with recursive function calls.
We're not going to look at some
of the more advanced cases of
how compilers can optimize code really well,
but you should know it is rather
expensive to push timewise,
to push something onto the stack or to pop it off.
We're going to look at an instance where
compiler might choose to do it a different way.
>> For now, we're going to look at the difference between
>> a standard call and a C declaration call.
We can actually do both in the same program
with instructions right next to each other.
Then I'm going to show you the different ways that data
can be put onto the stack as
just a preference thing by different compilers.
We can't solve some differences between
GCC in Visual Studio last time.
I will show those really quickly again this time.
Here we have our Hello World function again.
This time we're going
to have two variables that are going to be
passed in to the printf function along
with another parameter, the format string.
It's just going to print Hello World.
Then right under that is
API function call Hold MessageBoxA,
which just pops up a message box.
We're going to just debug this. I build it.
I'm going to go and generate the disassembly.
[NOISE] Look at it side-by-side if we want to.
Up here, we see push EBP,
move ESP into EBP and sub ESP.
These are the beginnings of most functions.
Most, or what we would call procedures or sub-routines,
is what the old terminology is for it.
To push EBP, takes
the current base pointer
and then pushes it onto the stack.
It's saving that value.
Then it takes the current ESP to the top of the stack,
and it puts it to EBP.
What you're essentially saying is,
I am going to save where
my current EBP is and then wherever my ESP is,
that's my new base of the frame.
That's where I'm going to start counting my values now.
As soon as this move happens, EDP and ESP,
are the same value,
they're pointing to the same location in memory.
Moved doesn't mean, move isn't what
>> you might think of it in terms of copy paste,
>> but move is more like copy.
It doesn't zero out anything.
When you have these two values equal the same thing,
you have a stack frame of no size.
Then we're subtracting 0D8 hex bytes from ESP.
We're extending our stack.
Now, we have D8 hex bytes
available on the stack that we can
use for storage of variables.
Mostly local variables.
That's pretty much [LAUGHTER]
only local variables should be stored on the stack.
What does it do then? There's a push, push, push.
It's saving more registers onto the stack.
This is another type of convention.
It's not call your caller convention.
It's the compiler deciding that
the callee should save the registers.
The code cannot rely on the fact that,
if it makes a call instruction,
that it's registers will still be the same values.
These are general purpose registers
like we were talking about last time.
That means they can be used by the code for anything.
You really shouldn't think
that if you call
something or jump somewhere or do something,
that the registers will
still be the same value afterwards.
A lot of compilers use
this convention where it saves the registers.
It's going to mess with on the stack.
Then right before is going to exit,
it'll pop them back off.
If you go down to the bottom, we
can see that it's popping
those original three values back into their registers.
It doesn't know what's in those registers.
It may not matter, it may not
care is just a safety thing.
Just in case the code calling this one main,
was relying on the fact that it could still use
those registers, which it shouldn't.
It's saving that. It's just being good.
Code up compilers may optimize this out.
It's convention and the X86 official manual by Intel,
they say do not
expect that these registers will be saved,
but compilers try to save them anyway.
What's it doing next? Those LEA,
which is load effective address.
This is really a shortcut,
use a lot of time to generate,
to do some quick math.
Because you can combine multiple additions,
subtraction, multiplication operations together,
and it's pretty fast.
In this context,
it's saying, ''What is EBP minus my stack size?''
Then it grabs that location,
and then puts it,
or it grabs the value at
that location and puts it into EDI.
What's it using for? I don't really know just yet.
>> The next instruction, is it puts the value of
36X into the ECX,
and it's moving this value into EAX.
I know what this is now because since I built
this program using debug right up here,
it's adding some extra safety net type code.
If this code is executed by accident,
if we have a buffer overrun
or something goes wrong,
and the EIP starts
executing the stack code, it'll break here.
CC is the instruction INT 3,
which is interrupt 3,
which is software interrupt,
which means it will signal with
debugger that something has gone wrong.
This is a repeat string instruction.
That has to do with EDI,
or that has to do
with the destination register and the source register.
We can see here that
it's going to use the value at this address.
Then whatever ESI is
pointing to is generally
what the string operation will use,
or generally the value in ESI and EDI are
what this string instructions do.
These repeat instructions act as a loop.
Not a whole lot of compilers will produce
these instructions because they
like to have more control,
or they'd like to manipulate the logic
and their loops a bit more,
but the string operations were
>> made by Intel because they
>> noticed a lot of the processing power
their chips are doing were string operations.
They weren't that much
faster than the other ways compilers
figured out how to do them.
Not that common, but you still see them
>> every once in a while.
>> We can see hello being stored into var1.
That's what this instruction down here,
where it's basically storing var.
There's a reference here.
This is a memory reference
to wherever this string is stored in memory.
It's storing that address
>> into location of variable one.
>> I'm doing the same for variable two.
Here is really what I wanted to show you.
printf is taking three parameters.
The first one is going to be var2.
It's going to do this maneuver,
where it's going to move
the location into EAX and then push EAX.
This is a string,
and a string is pretty much a character array.
At least if we're keeping it simple,
it's a character array of one byte ASCII values.
When we push from right
to left and we're pushing it in the opposite order,
we're going to go from var2 to var1.
We could have used EAX again,
but the compiler decided to store var1 in ECX,
and then it pushes ECX,
and then it pushes this last value.
This memory value must contain
a reference to this format string.
Then this call is performed,
where printf is then executed.
Then right after it,
we see an add ESP and 0C.
0C was the damage done to the stack by this push,
this push, and this push.
That was 12 bytes that were push,
four here, eight here, 12 here,
and 12 in hex is c 0X,
0X, 0c is the standard way of writing it.
If you also,
if you put an H at the end of it,
we know it's hex.
That was C dackle.
This is pretty common.
This is the standard C libraries that it's using.
That's the calling convention.
Standard call made by
Microsoft and pretty much only used by Microsoft.
Not really standard at all,
as it's called it, does the opposite.
For this line of code,
we're going to do something similar.
I don't know why that instruction was produced.
It's going to do a push.
That was this null byte or zero.
Then it's going to do another push
for the reference to the string.
Another push for the reference to the string.
Reference to the string. I'm going
to do another push as a reference to this zero.
Then it's going to do a call.
Now, this call looks a little weird.
That's because it's stored
the location of this function at this location.
This ds is data segment and dword means 32-bit,
and PTR means pointer.
It's a 32-bit pointer in the data segment.
This is part of our IAT or our import address table.
If you don't know what that is, that's okay.
It's part of the PE file structure. It is important.
>> You should get to know it down the line.
>> For now, just know that
the location of this function is stored here.
We can see it a bit more elegantly
>> and here in a minute.
>> Afterwards, there is no add ESP.
This function, this message box,
a function, cleaned up the stack.
It at some point did
a minus 16 bytes or it
added 16 bytes to ESP to clean it up.
You might be curious to see what other code is here.
Your compilers will insert code for you.
A lot of that has to do with performance
or security checks to
make sure the stack isn't corrupted or something,
or to set exception handlers,
SEH handlers, especially in debugging code,
because I built it in debug mode.
It will produce extra security checks
and wrapper functions and to just double check things.
Then this last instruction,
this instruction is very, very common.
You should really get to know it, XOR.
XOR, EAX,
EAX will make EAX zero.
The return value is kept in EAX.
Returns zero means turn
EAX to zero and then use the right function,
which is down here.
Here's the pop, pop,
pop that we saw earlier.
Another double-check to see
make sure that the ESP is correct.
While we were in debugging mode,
we're trying to figure out why something isn't
working like a particular function,
third-party libraries aren't working,
you can work out maybe why.
As I was saying, when I mentioned this CC bytes before,
these are the instructions the CC disassembled.
If you looked at this in a hex editor,
it'd be just be CC, CC,
CC, CC, CC, CC, CC and etc.
This is also in case our code kept executing beyond
this return value or maybe a jump
was off and jumped down here or something like that.
It happens from time to time.
If we want to look at this
in a disassembler without an IDE,
we can with IDA,
if we didn't have the source code.
>> We often don't for malware,
we would find this very useful.
I'm going to go hello.
There's a project we made last time.
I'm going to go to debug
because that's what we were just looking at.
Disassemble it, the
same that it noticed
that there is some debugging information in there.
Does it want to retrieve the file?
I going to say no because
we usually don't have that file,
I'm going to say no, I don't really
like the proximity review.
This might seem pretty innocent.
You might click through it and try to find
where your few lines of code were.
Remember, when I was
saying there's a lot of added code,
a lot of protections built-in for the stack,
for performance to help debuggers, this is it.
Last time we went and viewed the strings,
and found the strings that we knew were referenced.
I'm going to show you another technique that's
very popular with reverse engineers and probably,
the most common technique, is that we
go view the imports.
These are all the functions that
our program has requested to call.
You might look at this and you're just like, "Hey,
I didn't call that function query
performance counter," or get
current thread id or any of that.
No, that's one of the debugging code and
just other boilerplate code that Microsoft
will insert in there when it's inserting
its C libraries or whatever else.
We call the function MessageboxA.
I'm going to go there, I'm going to
see that it's at that location.
If I hit "X,"
I can see where else
this memory address is referenced in memory.
Overall, on the right here,
you'll be able to see
that it's referenced in two places,
but if I want to get a list of all of them,
I can hit "X." I can see that it's called right here.
These two values are the same address.
If you look over at type,
one is a pointer and the other one is it's being read.
That's what the R stands for and the P stands for.
Below it, there's a jump,
>> something that is jumping there.
>> We're just going to go with where we
know this thing is being called.
If you remember, this function was
a thing built into this program by
the compiler and it was like check ESI
or ESP or whatever it was checking.
It was checking something about the stack
but we pretty much get
the same instructions that we
saw with the same CC values,
the same push, push,
push, push, push, push.
IDA doesn't actually know what the stack was used for.
These weren't assembly instructions,
but it's figured out that based on how
the assembly was referencing
the various values in the stack,
it said, "Okay, there's probably
three local variables in here."
It got that right, sometimes it gets it
wrong, especially with arrays
but it also figured out that
this function was MessageboxA and it said,
"Okay, MessageboxA according
to Microsoft, has these parameters.
It knows it's calling them or it's going
>> to push them onto the stack
>> from right to left in reverse order."
>> It said, "What were the last four pushes?"
It's like, the parameter to that is u type.
The parameter to this is caption,
the parameter to this is text,
and the parameter to this is window handle.
It's very kindly provided hints as to what
the parameters of this MessageBoxA
is but be warned that
sometimes the analysis gets a little messed
up and it might be a little off.
These are helpful, but you should
not just completely rely on them.
It's also very helpful that it
put a little comment here and said, "Hey,
this is referencing a pointer
and that pointer is pointing to an ASCII string."
It's going to guess that this is
a string and it names it
A for ASCII and then whatever the string is.
That's very helpful.
We can also rename things in IDA.
If I click this and hit "N,"
I can rename this variable to var 1.
I can rename this variable to var
2 and then I'm going to just name this format string.
I know this is printf.
That's what it said
in my debugger and that's what said in my source code,
but this is showing up as sub,
short for subroutine and then
a number which is the memory address it's going to.
I can click on it and then scroll
down and get a peek of what's there.
It looks like there's just a jump there.
I can follow this jump by double-clicking on
the label and I can find all this other stuff.
I'd like to highlight the calls and jumps,
and I can see that it's
calling into several other things.
I'm just thinking, is this printf?
Is this something else?
As it turns out,
Microsoft doesn't actually include
the C libraries that are standard.
Microsoft implemented its own C runtime libraries.
Usually, they're bundled with
visual C runtime libraries or
VC runtime or whatever.
There's a lot of programming
that libraries out there for programmers use.
They're pretty standard, but this is not it.
This is more debugging code
and it's also more code to check certain things.
The parameters that you gave printf were safe
>> because they have been vulnerabilities,
>> but they can't change the functionality
of standard library
calls because maybe that
might break old software if you try to recompile it.
We know somewhere down this line there's going to be
a printf or equivalent thereof.
I happen to know what exactly this function call is.
I'm going to go over and look in the imports tab.
It's going to be underscore underscore
something because that's generally what they name it.
I can search by hitting
Control F and then just type in print,
and I can see it right here towards the bottom,
but I'm going to type in print anyway.
There's two function calls,
__stdio_commonvfprintf,
and then above it is sprintf_s. The compiler is
doing its best to protect us from
being dumb programmers, and that's very nice.
We can see if we hit "Escape,"
we can go back in
our analysis and we can get to our original stuff.
You'll notice that this function wasn't named main.
With the X, you can say,
what has a reference to this location?
You can see there's a jump.
What has a reference to this location?
Jump further back, what
has a reference to this location,
and then you can keep going like that.
>> Oops, excuse me.
>> Meant to click that, hit "X" on that,
and see what calls that function.
We can see there's more checking code for argv,
argc, and preparation code for us,
and we can jump to see what call is this,
and we can see that
we are in another function called this.
If I was curious to see what function calls
this function made,
I can say I can go under view and then graphs,
or I can right-click and say xrefs graphs from.
I can see that this function will call this function,
and then this function, and then this function.
That can be useful if you're trying to get
an idea of what a function
does because you can use
this graphing recursively and you can say,
"Okay, I know
this function has a bunch of functions it calls.
What are they? How many of them are libraries?"
Because the pink labeled function calls or
function names are those libraries or functions
that IDA has referenced or IDA has recognized.
If we wanted to see this actually
executing and we didn't have the original source code,
we can always crack open all the debugger.
Sorry, hello.exe.
>> Here Olly is loading in the libraries
>> and analyzing them.
>> We might not always know exactly where our code is.
Just similarly to us
not knowing where this code was in IDA.
We might not know where the code we want is in OllyDbg.
We can do the same thing where we can right-click and
then search for all inner module calls.
That basically looks for call instructions.
We can find the call instructions to MessageBoxA.
We could go there and see
the same push call and set a breakpoint.
We can do that with F2.
This will set a soft breakpoint.
Behind the scenes it's actually putting
an INT 3 instruction there or a CC byte.
Then when this code gets executed,
it will stop there and then the debugger will take
over and then replace
that byte with the original byte that was there,
and then begin to
execute as if nothing had ever happened.
I'm going to put a breakpoint here where the entrance
to this main function
is which Olly was kind enough to recognize.
I'm going to remove this breakpoint,
I'm going to hit "Play" and it ran,
and it has now stopped at my breakpoint.
Now, I can do the same step
over that I could with Visual Studio's debugger.
Now, this is important.
Let me start over.
[NOISE] I say yes.
My breakpoint is still there,
>> so I'm going to hit "Play" and
>> it's going to go breakpoint.
>> The first instruction was push EBP,
the next one was yes,
move ESP and EBP.
As I was saying earlier,
this is important because
[NOISE] here the stack is going to be manipulated.
This thing that's at the top of the stack,
AKA a lower memory address,
is the return function from the previous caller.
The previous function that called this one said,
call 101780.
It pushed EIP onto the stack.
EIP was going to execute this instruction next,
but instead the call forced it to come to this address.
Now, I'm going to make my own stack here.
How do I do that safely? The first step
is, [NOISE] push EBP.
I save the current base pointer.
I'm going to move ESP to EBP.
We can see EBP was just overwritten here,
is now the same value as ESP.
[NOISE] Now, I'm going to make room on the stack.
Step over and sub,
and it took D8 bytes.
We can see here on
the stack is now allocated all of this for
us to use and local variables and
>> see how they're all filled with CCs.
>> That was the debugging code that
was added in because
we didn't say this was released try use.
This thing probably has
bugs and we're going to want to fix them.
Just in case any of this code gets executed,
the debugger will kick in and say,
oh, something wrong happened.
Now, we have a local stack frame.
[NOISE] Now, we're about to execute another function.
Now, we're saving the registers.
Push. It doesn't know what was in them, doesn't care.
It's just trying to make sure
that those values are now saved.
This is important, where the string
hello is being stored in a local variable on the stack.
We can do step over.
Now, we can see a reference to
the hello string is right here.
Step over again and now world is again being used.
Now, if we're debugging the release version,
these CC bytes would not be here.
Step over is going to push EIX,
it's going to push world and then it's going to push
the format string and Olly
debug was nice enough to say, "oh,
play tough? Okay, you at least to have to have
one parameter in this format string," because it
doesn't know how many parameters are
going to get pushed into
the printf function call or pushed as parameters.
Now, I'm going to make this call.
I'm just going to step over.
I could step in and watch,
actually do exactly what it says it's going to do.
I can even flip over to
the tab and say, oh, to print out.
We notice that the values are still on the stack here.
Now, as I step over,
>> the next instruction, did this add ESP 0C.
>> Now, the stack frame has decreased.
The data is still there on the stack but
the extended stack pointer ESP is now pointed here.
It's now pointing at lower. As far
as the compiler is concerned,
as far as all these instructions are concerned,
this is now the top of the stack.
These other values don't
matter, they don't care about them.
They're going to override them
>> if they do anything else,
>> they no longer exist.
I step over again.
It's going to push 0,
push hello world,
push alert, push 0,
and then call MessageBoxA.
Watch carefully because while
ESP is up here as soon as we're going to call it,
the callee function is going to clean up the stack.
[NOISE] New modules are
being loaded and Olly is
analyzing them as disassembling Olly code.
Those are some pretty big DLLs.
Give it a minute. Now, there's a pop-up
>> that's happened and that's what this function does.
>> I hit "Okay," now that function finished.
You can notice the stack has cleaned up.
It didn't need to do that that add ESP.
That's pretty much the major difference between
those calling conventions and
you should be aware of them.
I don't really care what
happens to the rest of this function.
I'm just going to terminate the process and
take a quick look at the release version.
[NOISE]
>> It's stopped in what looks to be
simply some slightly different boilerplate code.
Here's the CH exception handling code.
We're going to find the module call,
find references too,
search for in your module calls.
Go to MessageBoxA, see Hello World.
Looks like it's calling print up from
there. It looks about right.
[NOISE] Here's some more code
up here and separating the code is n three,
just in case something overran were supposed to go,
will say stop over
and we can see it doing this stack manipulation.
It didn't subtract nearly as much.
[NOISE] There it's doing some checking.
Here's where it's pushing the parameters to printf
and set special printf function
that Microsoft made, that's a little safer.
[NOISE] It does some more parameter
pushing for another little
function call it's going to make,
which is V, F,
print
f and it finished,
which means that printed to the screen.
Now, it's fixing up the stack [NOISE] and it's done.
You'll notice it says add ESP 06
here [NOISE] and it's cleaning up the stack.
Now, we're about to start
pushing for the next function call,
which is MessageBoxA.
Push call and I'm going to switch
over to wherever made the call there it is.
Then it's ready to end this function,
which is return 0,
which is easy quick way to do it is
XOR EAX, as I said before.
I'm pretty much done with this.
I'm just going to exit out.
Quickly in Cygwin,
[NOISE] we notice that
the compiler changed in Visual Studio from
what functions it was calling based on the release and
what wrapper functions ahead
and Cygwin uses the GCC compiler,
the GNUC compilers with a stands for and
[NOISE] it will produce different code as well,
every compiler has its own little
[NOISE] programmers that made it, so it's important.
Here we demonstrated that I've compiled this
>> program just like it did in Visual Studio.
>> Now, I'm going to go find
that program Cygwin, home, Sean, a.exe.
>> I'm going to crack it open in IDA and
>> see what code it produced.
>> [NOISE]
Cygwin with GCC
answered a lot of extra code.
This looks to be some parameter checking,
but I'm going to go and find
the code that I'm most interested in.
I see MessageBoxA right there.
I'm going to go and hit "X". I'm going say,
okay, it's when we reference in two places.
One is a call instruction
and the other is a move instruction.
The call instructions, what I'm interested in.
This is something that
I really want to show you in that,
like I said, push instructions
are very expensive time-wise.
Some compilers have moved away from them.
The GCC compiler in this case,
when instead of pushing onto the stack,
is simply does a move.
It's moves the variable
onto the stack, because you can do that.
You can just move your register or the pointer to
the string and the pointer is stored in
part C. You can just move that into the stack.
This way it doesn't have to push,
but it does have to make sure the stack is
the right size by time it calls print
f. The compiler is
being a bit more
advanced and how it's trying to do something.
Similarly, when it calls MessageBoxA,
it does another little trick,
which malware will do frequently,
in which they'll have a function pointer
and store that function pointer in
a register and then call that register.
This makes it very hard for disassemblers to
figure out what functions are being
called from where unless
it's more intelligent like IDA Pro.
Even then IDA Pro will frequently
not catch the soul trick.
Afterwards, you'll see sub ESP.
It's not cleaning up after
this function call because
MessageBoxA will clean up after itself.
It's cleaning up for all the instructions before it.
These two function calls.
GCC is trying to be a bit faster than
Microsoft Visual Studio compiler.
That's it for this demo.
Thank you for watching we covered
standard code versus cdecl.
We looked at different ways code is
generated and we steps through a lot of assembly there.
We talked about how
compilers will choose to do certain things.
How in one instance,
a lot of push instructions for use to get data onto
the stack and then in another instance with GCC,
it'll just move the data onto the stack
instead of pushing and then we'll
clean up the stack when it's done with it.
In the next video,
we're going to do some stack analysis on
some actual malware. Hope to see you there.