Profiling, or OSINT (open source intelligence), is an art. Private investigators have been doing it for years now but, it has just started to show real promise in application to Penetration Testing and Red Team Testing. A lot of work has been done recently by Chris Gates and Chris Nickerson on bringing it into the security world.
OSINT gathering is a far more manual process than general profiling. It is usually not included in a regular pentest or assessment, but sometimes is included for an extra fee (from what we’ve seen) .
Why OSINT? It opens up your attack surface and gives you a sprinting start before even sending any packets directly to your target. Just entering in some of the RS engineers' data i was able to get a list of previous addresses, blogs, twitter names, emails, phone numbers, parents names, news articles, etc. I'd say that's useful.
For a place holder, until we get done with a nice framework, we have compiled a list of links for OSINT gathering web sites we use, by target type; People and Organizational Targets and Infrastructure Targets :
- Infrastructure:
- Netcraft (Uptime Survey, server info)
- Domain Tools (Whois Lookup and Domain info)
- Centralops.net (traceroute, nslookup, automatic whois lookup, ping, finger)
- Hackerfantastic.com ( GeoIP, whois, host, dig, blacklists, ping, traceroute & nmap)
- whois.webhosting.info (WHOIS and Reverse IP Service/virtual hosting info)
- MSN IP Search
- SSL Labs – Projects / Public SSL Server Database – SSL Server Test
- SHODAN – Computer Search Engine (indexed port scans and banner grabs)
Don't forget to combine this kind of "research" with great tools like wikto, metagoofil, SEAT, FOCA, theHarvester, and Maltego. We also suggest you listen/watch to some of the collective presentations and video from Chris Nickerson and Chris Gates (btw Dale Pearson is the man for setting up the Brucon videos last year).
Also, i had been working with some browser scripting earlier and i made a quick OSINT script in iMacro. iMacro can record and script up your day to day browser activities. Sometimes this is easier to do than coding something in Ruby or Python due to the AJAX nature of these people search sites. Download the firefox plugin for iMacro here then make a new macro with the below code:
VERSION BUILD=6700624 RECORDER=FX
PROMPT Please<SP>enter<SP>a<SP>target<SP>FIRST<SP>NAME !VAR1
PROMPT Please<SP>enter<SP>a<SP>target<SP>LAST<SP>NAME !VAR2
PROMPT Please<SP>enter<SP>a<SP>target<SP>COMAPNY !VAR3
TAB T=1
URL GOTO=http://www.spokeo.com/
TAG POS=1 TYPE=DIV ATTR=ID:search_bar
TAG POS=1 TYPE=INPUT:TEXT ATTR=ID:search_input CONTENT={{!VAR1}}<SP>{{!VAR2}}
TAG POS=1 TYPE=A ATTR=ID:search_submit
TAB OPEN
TAB T=2
URL GOTO=http://www.123people.com/s/{{!VAR1}}<SP>{{!VAR2}}
TAB OPEN
TAB T=3
URL GOTO=http://www.spoke.com/
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:quickSearchForm ATTR=ID:srName CONTENT={{!VAR1}}<SP>{{!VAR2}}
TAG POS=1 TYPE=TD ATTR=TXT:A<SP>lot<SP>of<SP>people<SP>want<SP>to<SP>share<SP>their<SP>professional<SP>info.
TAG POS=1 TYPE=BUTTON ATTR=ID:c_searchbutton-button
TAB OPEN
TAB T=4
URL GOTO=http://www.spoke.com/
TAG POS=1 TYPE=SELECT FORM=NAME:quickSearchForm ATTR=ID:companySearch CONTENT=%true
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:quickSearchForm ATTR=ID:srCompany CONTENT={{!VAR3}}
TAG POS=1 TYPE=BUTTON ATTR=ID:c_searchbutton-button
TAB OPEN
TAB T=5
URL GOTO=http://pipl.com/
TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:/search/ ATTR=ID:name2 CONTENT={{!VAR1}}
TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:/search/ ATTR=ID:name1 CONTENT={{!VAR2}}
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:searchForm ATTR=VALUE:<SP><SP><SP><SP>Search<SP><SP><SP><SP>
TAB OPEN
TAB T=6
URL GOTO=http://www.zabasearch.com/
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:form1 ATTR=ID:sname CONTENT={{!VAR1}}<SP>{{!VAR2}}
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:form1 ATTR=NAME:submit&&VALUE:Free<SP>People<SP>Search
TAB OPEN
TAB T=7
URL GOTO=http://www.searchbug.com/peoplefinder/
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:peopleForm ATTR=NAME:FNAME CONTENT={{!VAR1}}
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:peopleForm ATTR=NAME:LNAME CONTENT={{!VAR2}}
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:peopleForm ATTR=VALUE:Search
TAB OPEN
TAB T=8
URL GOTO=http://addictomatic.com/topic/%22{{!VAR1}}<SP>{{!VAR2}}%22
TAB OPEN
TAB T=9
URL GOTO=http://addictomatic.com/
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:searchform ATTR=ID:q CONTENT={{!VAR3}}
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:searchform ATTR=ID:search
TAB OPEN
TAB T=10
URL GOTO=http://socialmention.com/search?t=all&q=%22{{!VAR1}}<SP>{{!VAR2}}%22&btnG=Search
TAB OPEN
TAB T=11
URL GOTO=http://entitycube.research.microsoft.com/result.aspx?vtr=0&q=%22{{!VAR1}}<SP>{{!VAR2}}%22&action=keyin
TAB OPEN
TAB T=12
URL GOTO=http://entitycube.research.microsoft.com/result.aspx?vtr=0&q=%22{{!VAR3}}%22&action=keyin
TAB OPEN
TAB T=13
URL GOTO=http://www.yasni.com/{{!VAR1}}+{{!VAR2}}/check+people?sh