Sam is right. The problem is there because of the current text filtering logic:
1. Forum module calls format_text() when displaying the post message
2. The message text is converted into HTML by s() or markdown_to_html() or text_to_html() according the text format used for the post
3. The message text in HTML format is cleaned up by clean_text()
4. The chain of filters is applied to the message text
Now, when the post is in FORMAT_MOODLE (as in the case of attached example), text_to_html() is called in step 2. This function calls convert_urls_into_links() and so GeSHi filter has no chance to get to the original text without <a>
From my point of view, there is the only way how to get around this: get rid of convert_urls_into_links(), move it from core and transform it to a normal filter. Then admin would just set it as the last element in the filter chain and GeSHi would work correctly (tested on my own machine).
There is an issue, though. Currently, filters are not informed about the format in which the text was originally written - because at the moment they can be applied, the text is in HTML only. Therefore the new filter_urltolink would apply to all other formats, too. That breaks backwards compatibility as only URLs in FORMAT_MOODLE texts are converted now. But it is not that hard to extend our current filters API and provide them the information of the format.
Please note that the same logic should apply to replace_smilies() - it should be converted into ordinary filter, too.
I'll be happy to work on this. Should not take more than one day as the code and unit tests are already written.