User Controls
Hello Shodan.io API tech support?
-
2017-03-17 at 8:55 PM UTC*Brr Brr* *CLICK*
"Moshi moshi shodan desu?"
I know my usertitle says Tech Support but even tech support needs tech support sometimes ya dig?
So i was bullshitting around with the shodan API and i ended up writing a CLI, ya rly, and it doesn't matter shodan already has a CLI my CLI is speshull because i am writing it. So i am 200+ lines deep and i am testing out some of my functions that do the things that they do, looking for stuff on shodan and shit.
Now this works.
import shodan
from blessings import Terminal
t = Terminal()
api = shodan.Shodan(SHODAN_API_KEY)
def service():
global api
print "\n[" + t.green("+") + "]Please provide your search query."
query = raw_input("\n<" + t.cyan("SERVICE") + ">$" )
try:
result = api.search(query)
print
for service in result['matches']:
print service['ip_str']
print
except Exception as e:
print "[" + t.red("!") + "]Critical. An error was raised with the following error message.\n"
print e
print "\n[" + t.green("+") + "]Defaulting to main menu."
main()
Before you judge me for using a `global` statement i will have you know i have my menu in a function in a loop and just adding a `global` statement is way easier than conditional return statements, but feel free to submit a pull request when it's done.
Ok so back to business my function above gives me a nice list of IPs if i search for "routers" or whatnot. Now i have another function and it's purpose is to retrieve information on a specific IP and it looks like this.
def host():
global api
print "\n[" + t.green("+") + "]Please provide your search query."
query = raw_input("\n<" + t.cyan("HOST") + ">$" )
try:
result = api.host(query)
print
for term in result['matches']:
print term['value'] # Error, string indices must be integers?
print
except Exception as e:
print "[" + t.red("!") + "]Critical. An error was raised with the following error message.\n"
print e
print "\n[" + t.green("+") + "]Defaulting to main menu."
main()
As you can see in the comment, i am getting a "string indices must be integers" error. Also when i remove the weird string things from my for loops i get this as result:
# Terminal output
region_code
ip
postal_code
country_code
city
dma_code
last_update
vulns
latitude
tags
area_code
country_name
hostnames
org
data
asn
isp
longitude
country_code3
ip_str
os
ports
So i thought well ok, i am getting attributes of the search result but not their actual values or as Shodan likes to put it "facets" because the docs tell me "use facets bro it gone be fine".
So this is how shodan proposes i use facets.
import shodan
import sys
# Configuration
API_KEY = 'YOUR API KEY'
# The list of properties we want summary information on
FACETS = [
'org',
'domain',
'port',
'asn',
# We only care about the top 5 countries, this is how we let Shodan know to return 5 instead of the
# default 10 for a facet. If you want to see more than 10, you could do ('country', 1000) for example
# to see the top 1,000 countries for a search query.
('country', 5),
]
FACET_TITLES = {
'org': 'Top 10 Organizations',
'domain': 'Top 10 Domains',
'port': 'Top 10 Ports',
'asn': 'Top 10 Autonomous Systems',
'country': 'Top 5 Countries',
}
# Input validation
if len(sys.argv) == 1:
print 'Usage: %s <search query>' % sys.argv[0]
sys.exit(1)
try:
# Setup the api
api = shodan.Shodan(API_KEY)
# Generate a query string out of the command-line arguments
query = ' '.join(sys.argv[1:])
# Use the count() method because it doesn't return results and doesn't require a paid API plan
# And it also runs faster than doing a search().
result = api.count(query, facets=FACETS)
print 'Shodan Summary Information'
print 'Query: %s' % query
print 'Total Results: %s\n' % result['total']
# Print the summary info from the facets
for facet in result['facets']:
print FACET_TITLES[facet]
for term in result['facets'][facet]:
print '%s: %s' % (term['value'], term['count'])
# Print an empty line between summary info
print ''
except Exception, e:
print 'Error: %s' % e
sys.exit(1)
Fine whatever, so i add facets for my search.
But it doesn't fucking work. Also `result = api.host(query)` and `result = api.search(query)` is on purpose. Because you need other syntax to indicate you want to search on a host IP instead of a string like "routers" And i think this is wherein lies the problem, but if "host" part of the method uses facets as well then why is it not working when i add facets? Even if i just copy pasta the facets my terminal outputs when i run the second function? -
2017-03-18 at 8:08 AM UTCBro what is this shit, it's like reading a foreign language. Fuck, I thought dwarf fortress was next to indecipherable
-
2017-03-18 at 8:38 PM UTC
result = api.host(query)
for term in result['matches']:
print term['value'] # Error, string indices must be integers?
The issue here is the way dictionary iteration works. Consider this code:
d = {
'foo': 41
'bar': 42
}
for key in d:
print key
# Outputs:
# foo
# bar
A for loop over a dictionary iterates on keys. If you want values you can either do `print d[key]` in the example above, or if you want something more idiomatic you can:
for key, value in enumerate(d):
print "%s: %s" % (key, value)
In any case, the issue is that "term" is a dictionary key, not its value. -
2017-03-18 at 10:07 PM UTC
Originally posted by Lanny
result = api.host(query)
for term in result['matches']:
print term['value'] # Error, string indices must be integers?
The issue here is the way dictionary iteration works. Consider this code:
d = {
'foo': 41
'bar': 42
}
for key in d:
print key
# Outputs:
# foo
# bar
A for loop over a dictionary iterates on keys. If you want values you can either do `print d[key]` in the example above, or if you want something more idiomatic you can:
for key, value in enumerate(d):
print "%s: %s" % (key, value)
In any case, the issue is that "term" is a dictionary key, not its value.
I'll take the idiomatic solution thank you.
for key, value in enumerate(result):
print "%s: %s" % (key, value)
But alas, my terminal outputs the following.
0: region_code
1: ip
2: postal_code
3: country_code
4: city
5: dma_code
6: last_update
7: vulns
8: country_name
9: tags
10: area_code
11: latitude
12: hostnames
13: org
14: data
15: asn
16: isp
17: longitude
18: country_code3
19: ip_str
20: os
21: ports
I tried making it, `for key, title, value in enumerate(result)` but then it says it needs more than two values to unpack :/ -
2017-03-18 at 10:08 PM UTC
-
2017-03-18 at 10:10 PM UTChmm, maybe the python binds are doing something with the data, I was just looking at the api docs. Can you do `import json; print json.dumps(result)` and post the output?
-
2017-03-18 at 10:25 PM UTC
Originally posted by Lanny hmm, maybe the python binds are doing something with the data, I was just looking at the api docs. Can you do `import json; print json.dumps(result)` and post the output?
I certainly can, and it looks like you are right. So now what? Lol.
{"region_code": "CA", "ip": 2899907534, "postal_code": "94043", "country_code": "US", "city": "Mountain View", "dma_code": 807, "last_update": "2017-03-08T07:29:12.054739", "vulns": ["!CVE-2014-0160"], "latitude": 37.41919999999999, "tags": [], "area_code": 650, "country_name": "United States", "hostnames": [], "org": "Google", "data": [{"hostnames": [], "hash": -1390436484, "asn": "AS15169", "timestamp": "2017-03-08T07:29:12.054739", "title": "302 Moved", "ip": 2899907534, "org": "Google", "isp": "Google", "transport": "tcp", "data": "HTTP/1.1 302 Found\r\nLocation: https://www.google.com/?gws_rd=ssl\r\nCache-Control: private\r\nContent-Type: text/html; charset=UTF-8\r\nP3P: CP=\"This is not a P3P policy! See https://www.google.com/support/accounts/answer/151657?hl=en for more info.\"\r\nDate: Wed, 08 Mar 2017 07:29:04 GMT\r\nServer: gws\r\nContent-Length: 231\r\nX-XSS-Protection: 1; mode=block\r\nX-Frame-Options: SAMEORIGIN\r\nSet-Cookie: NID=98=mCRHz4rgLbD7jQagTkslYGaOab5IqDI7w5K-ZaWk6rBVQvZhWlpLWoarNoYRtM9szbEns11bHOv_dOKEEeWM5Sj3jGiReGE_PXCiHYYpMrITePtuZx8Bv2-bbYEl6tr_; expires=Thu, 07-Sep-2017 07:29:04 GMT; path=/; domain=.google.com; HttpOnly\r\n\r\n", "port": 80, "html": "<HTML><HEAD><meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<TITLE>302 Moved</TITLE></HEAD><BODY>\n<H1>302 Moved</H1>\nThe document has moved\n<A HREF=\"https://www.google.com/?gws_rd=ssl\">here</A>.\r\n</BODY></HTML>\r\n", "location": {"city": "Mountain View", "region_code": "CA", "area_code": 650, "longitude": -122.0574, "country_code3": "USA", "latitude": 37.41919999999999, "postal_code": "94043", "dma_code": 807, "country_code": "US", "country_name": "United States"}, "deprecated": {"html": {"new": "http.html", "eol": "2016-12-31"}, "title": {"new": "http.title", "eol": "2016-12-31"}}, "domains": [], "http": {"redirects": [{"host": "172.217.19.206", "data": "HTTP/1.1 301 Moved Permanently\r\nLocation: http://www.google.com/\r\nContent-Type: text/html; charset=UTF-8\r\nDate: Wed, 08 Mar 2017 07:29:04 GMT\r\nExpires: Fri, 07 Apr 2017 07:29:04 GMT\r\nCache-Control: public, max-age=2592000\r\nServer: gws\r\nContent-Length: 219\r\nX-XSS-Protection: 1; mode=block\r\nX-Frame-Options: SAMEORIGIN\r\n\r\n", "location": "/"}], "title": "302 Moved", "robots": "User-agent: *\nDisallow: /search\nAllow: /search/about\nDisallow: /sdch\nDisallow: /groups\nDisallow: /index.html?\nDisallow: /?\nAllow: /?hl=\nDisallow: /?hl=*&\nAllow: /?hl=*&gws_rd=ssl$\nDisallow: /?hl=*&*&gws_rd=ssl\nAllow: /?gws_rd=ssl$\nAllow: /?pt1=true$\nDisallow: /imgres\nDisallow: /u/\nDisallow: /preferences\nDisallow: /setprefs\nDisallow: /default\nDisallow: /m?\nDisallow: /m/\nAllow: /m/finance\nDisallow: /wml?\nDisallow: /wml/?\nDisallow: /wml/search?\nDisallow: /xhtml?\nDisallow: /xhtml/?\nDisallow: /xhtml/search?\nDisallow: /xml?\nDisallow: /imode?\nDisallow: /imode/?\nDisallow: /imode/search?\nDisallow: /jsky?\nDisallow: /jsky/?\nDisallow: /jsky/search?\nDisallow: /pda?\nDisallow: /pda/?\nDisallow: /pda/search?\nDisallow: /sprint_xhtml\nDisallow: /sprint_wml\nDisallow: /pqa\nDisallow: /palm\nDisallow: /gwt/\nDisallow: /purchases\nDisallow: /local?\nDisallow: /local_url\nDisallow: /shihui?\nDisallow: /shihui/\nDisallow: /products?\nDisallow: /product_\nDisallow: /products_\nDisallow: /products;\nDisallow: /print\nDisallow: /books/\nDisallow: /bkshp?*q=*\nDisallow: /books?*q=*\nDisallow: /books?*output=*\nDisallow: /books?*pg=*\nDisallow: /books?*jtp=*\nDisallow: /books?*jscmd=*\nDisallow: /books?*buy=*\nDisallow: /books?*zoom=*\nAllow: /books?*q=related:*\nAllow: /books?*q=editions:*\nAllow: /books?*q=subject:*\nAllow: /books/about\nAllow: /booksrightsholders\nAllow: /books?*zoom=1*\nAllow: /books?*zoom=5*\nAllow: /books/content?*zoom=1*\nAllow: /books/content?*zoom=5*\nDisallow: /ebooks/\nDisallow: /ebooks?*q=*\nDisallow: /ebooks?*output=*\nDisallow: /ebooks?*pg=*\nDisallow: /ebooks?*jscmd=*\nDisallow: /ebooks?*buy=*\nDisallow: /ebooks?*zoom=*\nAllow: /ebooks?*q=related:*\nAllow: /ebooks?*q=editions:*\nAllow: /ebooks?*q=subject:*\nAllow: /ebooks?*zoom=1*\nAllow: /ebooks?*zoom=5*\nDisallow: /patents?\nDisallow: /patents/download/\nDisallow: /patents/pdf/\nDisallow: /patents/related/\nDisallow: /scholar\nDisallow: /citations?\nAllow: /citations?user=\nDisallow: /citations?*cstart=\nAllow: /citations?view_op=new_profile\nAllow: /citations?view_op=top_venues\nAllow: /scholar_share\nDisallow: /s?\nAllow: /maps?*output=classic*\nAllow: /maps?*file=\nAllow: /maps/api/js?\nAllow: /maps/d/\nDisallow: /maps?\nDisallow: /mapstt?\nDisallow: /mapslt?\nDisallow: /maps/stk/\nDisallow: /maps/br?\nDisallow: /mapabcpoi?\nDisallow: /maphp?\nDisallow: /mapprint?\nDisallow: /maps/api/js/\nDisallow: /maps/api/staticmap?\nDisallow: /maps/api/streetview\nDisallow: /mld?\nDisallow: /staticmap?\nDisallow: /maps/preview\nDisallow: /maps/place\nDisallow: /help/maps/streetview/partners/welcome/\nDisallow: /help/maps/indoormaps/partners/\nDisallow: /lochp?\nDisallow: /center\nDisallow: /ie?\nDisallow: /blogsearch/\nDisallow: /blogsearch_feeds\nDisallow: /advanced_blog_search\nDisallow: /uds/\nDisallow: /chart?\nDisallow: /transit?\nDisallow: /extern_js/\nDisallow: /xjs/\nDisallow: /calendar/feeds/\nDisallow: /calendar/ical/\nDisallow: /cl2/feeds/\nDisallow: /cl2/ical/\nDisallow: /coop/directory\nDisallow: /coop/manage\nDisallow: /trends?\nDisallow: /trends/music?\nDisallow: /trends/hottrends?\nDisallow: /trends/viz?\nDisallow: /trends/embed.js?\nDisallow: /trends/fetchComponent?\nDisallow: /trends/beta\nDisallow: /musica\nDisallow: /musicad\nDisallow: /musicas\nDisallow: /musicl\nDisallow: /musics\nDisallow: /musicsearch\nDisallow: /musicsp\nDisallow: /musiclp\nDisallow: /urchin_test/\nDisallow: /movies?\nDisallow: /wapsearch?\nAllow: /safebrowsing/diagnostic\nAllow: /safebrowsing/report_badware/\nAllow: /safebrowsing/report_error/\nAllow: /safebrowsing/report_phish/\nDisallow: /reviews/search?\nDisallow: /orkut/albums\nDisallow: /cbk\nAllow: /cbk?output=tile&cb_client=maps_sv\nDisallow: /maps/api/js/AuthenticationService.Authenticate\nDisallow: /maps/api/js/QuotaService.RecordEvent\nDisallow: /recharge/dashboard/car\nDisallow: /recharge/dashboard/static/\nDisallow: /profiles/me\nAllow: /profiles\nDisallow: /s2/profiles/me\nAllow: /s2/profiles\nAllow: /s2/oz\nAllow: /s2/photos\nAllow: /s2/search/social\nAllow: /s2/static\nDisallow: /s2\nDisallow: /transconsole/portal/\nDisallow: /gcc/\nDisallow: /aclk\nDisallow: /cse?\nDisallow: /cse/home\nDisallow: /cse/panel\nDisallow: /cse/manage\nDisallow: /tbproxy/\nDisallow: /imesync/\nDisallow: /shenghuo/search?\nDisallow: /support/forum/search?\nDisallow: /reviews/polls/\nDisallow: /hosted/images/\nDisallow: /ppob/?\nDisallow: /ppob?\nDisallow: /accounts/ClientLogin\nDisallow: /accounts/ClientAuth\nDisallow: /accounts/o8\nAllow: /accounts/o8/id\nDisallow: /topicsearch?q=\nDisallow: /xfx7/\nDisallow: /squared/api\nDisallow: /squared/search\nDisallow: /squared/table\nDisallow: /qnasearch?\nDisallow: /app/updates\nDisallow: /sidewiki/entry/\nDisallow: /quality_form?\nDisallow: /labs/popgadget/search\nDisallow: /buzz/post\nDisallow: /compressiontest/\nDisallow: /analytics/feeds/\nDisallow: /analytics/portal/\nDisallow: /analytics/uploads/\nAllow: /alerts/manage\nAllow: /alerts/remove\nDisallow: /alerts/\nAllow: /alerts/$\nDisallow: /ads/search?\nDisallow: /ads/plan/action_plan?\nDisallow: /ads/plan/api/\nDisallow: /ads/hotels/partners\nDisallow: /phone/compare/?\nDisallow: /travel/clk\nDisallow: /hotelfinder/rpc\nDisallow: /hotels/rpc\nDisallow: /flights/rpc\nDisallow: /commercesearch/services/\nDisallow: /evalua", "server": "gws", "host": "www.google.com", "html": "<HTML><HEAD><meta http-equiv=\"content-type\" content=\"text/html;charset=utf-8\">\n<TITLE>302 Moved</TITLE></HEAD><BODY>\n<H1>302 Moved</H1>\nThe document has moved\n<A HREF=\"https://www.google.com/?gws_rd=ssl\">here</A>.\r\n</BODY></HTML>\r\n", "location": "/", "components": {}, "sitemap": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<sitemapindex xmlns=\"http://www.google.com/schemas/sitemap/0.84\">\n <sitemap>\n <loc>https://www.google.com/edu/sitemap.xml</loc>\n </sitemap>\n <sitemap>\n <loc>https://enterprise.google.com/sitemap.xml</loc>\n </sitemap>\n <sitemap>\n <loc>https://www.google.com/gmail/sitemap.xml</loc>\n </sitemap>\n <sitemap>\n <loc>https://www.google.com/forms/sitemaps.xml</loc>\n </sitemap>\n <sitemap>\n <loc>https://www.google.com/slides/sitemaps.xml</loc>\n </sitemap>\n <sitemap>\n <loc>https://www.google.com/sheets/sitemaps.xml</loc>\n </sitemap>\n <sitemap>\n <loc>https://www.google.com/adwords/sitemap.xml</loc>\n </sitemap>\n <sitemap>\n <loc>https://www.google.com/drive/sitemap.xml</loc>\n </sitemap>\n <sitemap>\n <loc>https://www.google.com/docs/sitemaps.xml</loc>\n </sitemap>\n <sitemap>\n <loc>https://gsuite.google.com/sitemap.xml</loc>\n </sitemap>\n <sitemap>\n <loc>https://www.google.com/get/sitemap.xml</loc>\n </sitemap>\n <sitemap>\n <loc>https://www.google.com/flights/sitemap.xml</loc>\n </sitemap>\n <sitemap>\n <loc>https://www.google.com/admob/sitemap.xml</loc>\n </sitemap>\n <sitemap>\n <loc>https://www.google.com/ads/sitemap.xml</loc>\n </sitemap>\n <sitemap>\n <loc>https://www.google.com/business/sitemap.xml</loc>\n </sitemap>\n <sitemap>\n <loc>https://www.google.com/partners/about/sitemap.xml</loc>\n </sitemap>\n</sitemapindex>\n", "html_hash": 1604454775}, "os": null, "_shodan": {"crawler": "97b9d37f0484f45ce645307121c5c1ce0b3db578", "options": {"referrer": "01cdf2ee-6bd8-4b25-8783-e216f9e06ee9"}, "module": "http", "id": "01cdf2ee-6bd8-4b25-8783-e216f9e06ee9"}, "opts": {}, "ip_str": "172.217.19.206"}], "asn": "AS15169", "isp": "Google", "longitude": -122.0574, "country_code3": "USA", "ip_str": "172.217.19.206", "os": null, "ports": [443, 80]} -
2017-03-18 at 10:43 PM UTCI see, so there is no "matches" key on the host response, only on the search response. What you probably want is something like this:
result = api.host(query)
print 'Detail on %s:' % result['ip']
for service in result['data']:
print '\t- Running %s on port %d' % (service.get('product', 'unknown service'), service['port'])
So notice the general shape of the response: there are some keys that pertain to the host, like where it geolocates to, its IP address. Then there's a list on the "data" key that has an entry for each service running on that host. -
2017-03-18 at 10:44 PM UTCFuck this shithole.
-
2017-03-18 at 10:54 PM UTCOh yeah, that works, so if i wanted to get the geolocation as well would i use the service.get syntax as well? Preferably I would get geolocation, ports, services and vulns.
-
2017-03-19 at 1:23 AM UTC
-
2017-03-19 at 2:04 AM UTC
Originally posted by Sophie Oh yeah, that works, so if i wanted to get the geolocation as well would i use the service.get syntax as well? Preferably I would get geolocation, ports, services and vulns.
Geo info pertains to the host rather than the services it runs (like you can run http and ssh from the same host but they'd still have the same IP and still geolocate to the same place), so you'd use result[whatever] (like "city" or whatever keys you expect). Also worth mentioning that all dictionaries have the `.get()` method, `dict.get('foo')` is almost the same thing as `dict['foo']`, the difference is the former will return None (or its second arg, if supplied) if the key doesn't exist in that dictionary, whereas the latter will throw a KeyError, so in this case it's just a way of supplying a default value.The following users say it would be alright if the author of this post didn't die in a fire! -
2017-03-19 at 5:13 AM UTC
