Building Complete Call Graphs with Phoenix

By Alexander Schmidt

In my last post, I told you that we use Phoenix for building the WRK, which allows us to apply Phoenix' comprehensive set of analysis capabilities to the WRK. One particular analysis might be the construction of complete call graphs for functions interest.

Unfortunately, the build process of the WRK compiles each module separately and links it into a static library. As a final step, all static libraries will be linked together with pre-compiled libraries to the ntoskrnl.exe executable image. So building complete call graphs may be a problem, especially when a function calls or is called by a function within another module as Phoenix is only aware of functions within the compiled module.

But fortunately, Phoenix provides a solution to this problem!

Phoenix supports link-time code generation (LTCG), which is originally intended for whole program analysis. When using LTCG, Phoenix does not compile modules separately into COFF object files. It rather stores the intermediate language representation of that module in the object file or static library, respectively. Then, at link time, Phoenix is invoked again to compile all the modules at the same time, which allows global optimization or, in our case, enables us to provide a Phoenix Phase plug-in to construct a complete call graph of a function.

In order to enable LTCG with the WRK, you have to modify the provided makefiles a little bit. You must ensure that the compiler does not generate object code. This is done by the GL switch. As this is a target independent feature, add the /GL switch to the copts variable in file makefile.build, which is in directory base/ntos/BUILD. Next, we must notify lib.exe that it has to link itermediate language files into static libraries. For the record, lib.exe determines that on its own, but declaring that fact explicitly may reduce link time. So add the /LTCG switch to the LIBFLAGS variable of the same makefile (base/ntos/BUILD/makefile.build).

Next, we need to inform the linker that it has to invoke Phoenix before linking the static libraries together. This is done again by the /LTCG switch. So edit the makefile (not makefile.build!) in directory ntos/base/BUILD. I suggest here to modify the LINKFLAGS variable to contain the switch.

Finally, we need to inform Phoenix about what plug-in should be used. I recommend using the phx environment variable, which Phoenix will evaluate for command line options. As we want Phoenix to invoke the plug-in only at the link step and not for each module, we suggest to set the variable only for the linker. Modify therefore the kernelexe rule in build/ntos/BUILD/makefile. Below is a sample of our modification:

kernelexe:
        @set phx=-plugin:path\to\plugin\plugin.dll <More Options>
        $(LINK) $(LINKFLAGS) $(ntoslinkopts) -out:$(fullkernel).exe -map:$(fullkernel).map -pdb:$(fullkernel).pdb -entry:$(entrypoint) \
                $(hotpatch) PREBUILT\$(targ)\ntoskrnl.res $(OBJ)\ntkrnlmp.obj $(OBJ)\*.lib $(ntoswrklib) hal.lib $(fullkernel).exp $(bootlibs)
        $(LINKEDIT) -section:.rsrc,!d $(fullkernel).exe

The first line of this rule sets the environment variable phx. The -plugin option allows to specify the DLL that contains your Phoenix plug-in. You may or may not specify other options as well.

I wrote a plug-in which generates a call graph in the dot format for each function being compiled. The call graph contains both callers and callees of the compiled function. To enhance readability, I limited the depth of the graph to 4 levels for callers and callees. Potentially, you may construct a complete call graph, but my experience shows, that this generates graphs that are too complex to handle for dot. But you may experiment about that on your own 😉 . Click on this link to see the call graph that was generated for the MiReplaceWorkingSetEntry function. This function will become important in one of my next posts. So keep watching …

Comments