Debugging
of DLLs
Example 1: MessageBox
Example 2: wsprintf
Details and sources
OllyDbg 1.10 can debug standalone DLLs. Windows is unable to launch DLL directly, so OllyDbg uses small executable named loaddll.exe. This program is kept as a packed resource. If file you are trying to open is a dynamic link library, OllyDbg automatically extracts loaddll.exe and starts it, passing library name as a parameter.
With the help of loaddll, you can call functions exported by debugged library. I will explain this feature on the example of Windows' API functions MessageBox and wsprintf that reside in USER32.DLL.
1. Load DLL in the same way as ordinary .exe file. OllyDbg issues a warning:
Of course, we answer with "Yes". OllyDbg starts loaddll, loads library and pauses on a breakpoint that immediately preceeds the main window loop. This address is labelled as Firstbp. Then OllyDbg analyses DLL and displays its code. Note that Windows automatically execute DLL startup code when DLL is loaded into memory.
2. From the main menu, select "Debug|Call DLL export". The appearing dialog is non-modal, so you still have full access to all OllyDbg features. You can browse code and data, set breakpoints, modify memory and so on.
3. Select the function you want to call. We will begin with MessageBox. Note that this name is generic, in reality there are ASCII version MessageBoxA and UNICODE version MessageBoxW. Let's try with the second one. As we select it, rectangle to the right says: Number of arguments: 4. Analyzer determined that function ends with RET 10 and correctly recognized number of parameters. RET nnn is typical for functions that use PASCAL calling convention (parameters are passed on the stack, first parameter is pushed last, function removes parameters after call). Most Windows' API functions are PASCAL-style.
4. Set number of stack arguments. In our case this is not necessary, because OllyDbg already knows number of arguments in call to MessageBoxW. But, of course, you can override this decision anytime by clicking on the corresponding checkbox to the left.
5. Fill list of arguments. This dialog supports up to 10 stack parameters. Parameter is any valid expression that doesn't use registers. If operand points to memory, Dump window to the right from the argument displays contents of this memory. Loaddll.exe reserves 10 memory buffers, 1 K each, labelled as Arg1 .. Arg10, that you can freely use for any purpose. Additionally, dialog supports two pseudovariables: handle of parent window <Hwnd> created by loaddll.exe and handle of loaddll's instance <Hinst>. For your convenience, when you use Call export for the first time, OllyDbg adds them to history lists.
MessageBoxW expects 4 parameters:
- handle of owner window.
Here, we simply select <Hwnd>;
- address of UNICODE text in
message box. Select Arg2
and press Enter. Dump displays contents of memory buffer in hexadecimal
format. This buffer is initially filled with zeros. Right click on the
Dump and choose "Text|UNICODE (32 chars)" presentation. Select first
character and press Ctrl+E
(or, alternatively, choose "Binary|Edit" from menu). In the appearing
window, type "Text in box" or any other text to display;
- address of UNICODE title of message box. Select Arg3 and write "Box title" in UNICODE format to pointed memory;
- style of message box
as a combination of MB_xxx constants. OllyDbg knows them, type here
MB_OK|MB_ICONEXCLAMATION.
7. Select options. Hide on call means that dialog box should disappear from the screen when function executes. This option is useful when execution takes significant time, or if you set breakpoints. You can also close dialog manually. When called function finishes execution, OllyDbg will automatically reopen Call export. Pause after call means that debugged application will be paused after execution.
If everything is done correctly, dialog will look similar to this picture:
8. Call function by pressing Call.OllyDbg automatically backups all Dumps, verifies and calculates parameters and registers, removes dialog from the screen and then calls MessageBoxW. As expected, message box appears on the screen:
Bingo! Press OK. MessageBoxW returns and Call export reports success. Note that on return EAX contains 1. This is the numerical value of constant IDOK ("OK pressed"). This was simple, wasn't it?
1. Select the function. I hope, Call export is still open? Like MessageBox, wsprintf also has two forms: ASCII wsprintfA and UNICODE wsprintfW. We will play with its ASCII form. As wsprintf accepts variable number of arguments, it uses C calling convention. Main difference from PASCAL is that it is the responsibility of calling code to clean stack from parameters after call. C functions end with RET and Analyzer is unable to determine number of arguments.
2. Set number of stack arguments. wsprintfA has variable number of arguments; how many - depends on format string. Let's try the following call:
wsprintf(Arg1,"arg3=%i,
arg4=%08X",100,0x12345678);
As you see, we have 4 arguments, so click on checkbox "4".3. Fill list of arguments.
- First argument is a buffer. Choose <Arg1> and change dump format to ASCII (32 chars);
- Second argument is format string. Choose <Arg2> and change dump to ASCII (32 chars). Select first character, press Ctrl+E (binary edit) and type format string in ASCII field;
- Third argument is a decimal constant 100. By default, OllyDbg assumes hexadecimal format. Decimal point at the end of the constant forces decimal;
- Fourth argument is a
hexadecimal constant, just type it as is. OllyDbg accepts any
form: 0x12345678, 12345678h or simply 12345678;
Highlighted characters in dump of Arg1 are those modified by call. In register EAX, wsprintf returns number of characters in output string: 0x17 (decimal 23.).
Details and sources
loaddll.exe is a compact Win32 application written in Assembler. Have a look at its source code here. Execution begins at START. loaddll gets command line, skips name of executable (must be taken into double quotes!), extracts path to DLL and passes it to LoadLibrary. On error, it places pointer to error message on fixed location and exits with code 0x1001. On success, it creates simple main window and pauses on Firstbp. This breakpoint is set by OllyDbg on startup.All communication with OllyDbg is done through the 128-byte link area. This area must begin at address 0x420020 immediately after keyphrase. First several words contain addresses in loaddll.exe used by OllyDbg to set breakpoints and parameters, followed by address of function to call, contents of registers, number of arguments and arguments itself. Number of arguments is limited to 10. If argument is a pointer to memory, you can use 10 data buffers, 1 Kbyte each, named as Arg1, Arg2, ..., Arg10. These and some other names are exported and thus known to OllyDbg.
When loaddll passes main windows loop (WINLOOP), it constantly checks whether address of exported function in PROCADR is not 0. If this is the case, loaddll saves contents of ESP and EBP and pushes 16 zeros into stack. This is necessary to avoid crash if user specifies invalid number of arguments. Then it pushes arguments and sets registers. At address Prepatch there are 16 NOPs that you can use for small patches. If you need more space, you can jump to Patcharea 2 Kbytes long. Note that OllyDbg doesn't extract loaddll.exe from resources if file with this name already exists.
At CallDLL export is called. This command is followed by another 16 NOPs. Then routine saves modified registers and offset of ESP after call. If you supply invalid number of arguments to PASCAL-style function, OllyDbg will be able to report this error to you. Finally, loaddll restores ESP and EBP, zeroes PROCADR and breaks at INT3 at address Finished. When this point is reached, OllyDbg knows that execution is finished.
Treat LOADDLL.ASM as a freeware. I will not protest if you use this program as whole or in parts (without copyright) in your own programs. But do not dare to use the Green Bug (LOADDLL.RC) in projects not related to OllyDbg! That's all for now, enjoy!