I'd like to add two points to this discussion. First is that a key value I try to present to clients is that pen testing shows business impact. It lets a manager understand why security is important to the business. A list of vulnerabilities for IPs doesn't demonstrate quite the same impact as controlling some core business system. So successfully exploiting vulns is important to me.<br>
<br>Second, I see terribly insecure apps across enterprises all the time. They're niche products or internally developed that often sit on key systems. They usually don't have public vulns because they're internal or niche but if you sit down with them they're generally easy enough to break. So doing so is reasonable way to get into a fully patched system. It also makes you look good and reinforces security best practices like compartmentalization, defense in depth, etc. <br>
<br>So while I agree pen testers don't need to be exploit developers and it isn't a skill that's always needed, I'd add that it is one that can really turn a vanilla assessment into cool work.<br><br><br><div class="gmail_quote">
On Sun, Jul 13, 2008 at 3:03 PM, Thomas Ptacek <<a href="mailto:tqbf@matasano.com">tqbf@matasano.com</a>> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="Ih2E3d">> The problem I see with this is that people that can't write a simple<br>
> exploit also cannot to other very important tasks such as:<br>
> - Decide if a crash is exploitable at all<br>
<br>
</div>Plenty of people who can't write X86 assembly can discern whether a<br>
flaw allowed them to corrupt memory. Plenty of people who can write<br>
X86 assembly, like myself, are content to leave it at that: memory<br>
corruption bad. MUSTFIX.<br>
<div class="Ih2E3d"><br>
> - Make a judgement about the reliability of any exploits written<br>
<br>
</div>This is circular. Sure, if you write exploits, knowing how to do so<br>
reliably will in fact improve the quality of the checks you write for<br>
your company's scanner.<br>
<div class="Ih2E3d"><br>
> - Debug the crash to see what input caused the crash in a reasonable time limit<br>
<br>
</div>This isn't true. Basic investigative skills, of the sort possessed by<br>
many 2nd tier call center operators, coupled with the ability to<br>
generate malicious outputs, and you've got this one nailed. I agree<br>
it's important, so test for it.<br>
<br>
> - Discuss possible fixes intellegently<br>
<br>
What does ret-to-libc have to do with knowing how to manage sign bits,<br>
check multiplications, or bound copies?<br>
<div class="Ih2E3d"><br>
> - Apply knowledge of the crash to other areas of the program to ensure<br>
> that the bug isn't repeated and that the fix is in fact complete<br>
<br>
</div>It really sounds like you want to test people's ability to write<br>
fuzzers. Amen to that. I'm not sure where the shellcode comes in to<br>
it, though.<br>
<font color="#888888"><br>
--<br>
</font><div><div></div><div class="Wj3C7c">---<br>
Thomas H. Ptacek // matasano security<br>
read us on the web: <a href="http://www.matasano.com/log" target="_blank">http://www.matasano.com/log</a><br>
_______________________________________________<br>
Dailydave mailing list<br>
<a href="mailto:Dailydave@lists.immunitysec.com">Dailydave@lists.immunitysec.com</a><br>
<a href="http://lists.immunitysec.com/mailman/listinfo/dailydave" target="_blank">http://lists.immunitysec.com/mailman/listinfo/dailydave</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>Matthew Wollenweber<br><a href="mailto:mwollenweber@gmail.com">mwollenweber@gmail.com</a> | <a href="mailto:mjw@cyberwart.com">mjw@cyberwart.com</a><br><a href="http://www.cyberwart.com">www.cyberwart.com</a>